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FOREWORD 


Intelligence  collection  syst'ans  have  proliferated  over  the  past  several 
years,  increasing  in  complexity  and  in  volume  of  output.  However,  there 
has  been  no  corresponding  improvement  in  the  ability  of  intelligence  per¬ 
sonnel  to  analyze  this  flood  of  data.  The  US  Army  Intelligence  and  Security 
Command  (INSCOM)  studies  and  the  US  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences  (ARI)  research  indicate  that  improved  support  to  and 
training  of  analysts  are  necessary  to  effectively  utilize  the  increased 
collection  capability  and  satisfy  increasing  demands  for  intelligence 
within  current  personnel  constraints.  INSCOM  and  ARI  therefore  initiated  a 
joint  research  program  to  provide  improved  support  to  the  intelligence 
analyst.  During  early  discussions  of  the  issues,  it  became  clear  that 
any  procedural,  training,  organizational,  or  syston  changes  to  support 
analysis  will  be  effective  only  if  based  upon  a  detailed  understanding  of 
the  analysts’  role,  methods,  and  thought  processes  in  intelligence  produc¬ 
tion.  The  first  need  was  to  evaluate  and  describe  the  human  analytic 
processes  underlying  intelligence  analysis,  synthesis,  and  production. 

The  US  Army  Intelligence  and  Threat  Analysis  Center  (ITAC)  and  ARI  have 
successfully  applied  the  research  on  the  cognitive  bases  of  intelligence 
analysis  to  the  development  of  a  handbook  for  strategic  analysts  (ITAC  Report 
ATC-PP-2660-83)  ,  Given  the  growing  demands  upon  ITAC  resources,  it  is  vital 
that  new  ITAC  analysts  become  full  contributing  members  of  the  ITAC  team  in 
as  short  a  time  as  possible.  The  handbook  provides  new  analysts  with 
valuable  background  about  ITAC  as  a  work  environment  and  intelligence 
producing  organization.  Perhaps  most  importantly ,  the  handbook  also  emphasizes 
the  cognitive  tasks  of  analysis  and  the  development  of  skills  that  enhance 
one's  ability  to  think  logically  and  analytically. 

This  report  summarizes  the  background  research  that  led  to  the  development 
of  the  ITAC  Handbook.  First  it  describes  the  general  cognitive  model  of  intel¬ 
ligence  analysis.  It  then  identifies  cognitive  skills  required  of  successful 
analysts  and  relates  those  skills  to  the  performance  of  analytic  tasks  within 
the  context  of  threat  modeling.  Finally,  important  issues  related  to  the 
application  of  the  research  are  addressed.  This  report  should  be  very  useful 
for  the  development  or  evaluation  of  other  training  procedures  or  materials, 
analytic  procedures,  doctrine,  and  system  requirements  for  automated  support 
to  analysts. 
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THE  COGNITIVE  BASES  OE  INTELLIGENCE  ANALYSIS 


EXECUTIVE  SUMMARY 


Rcqu irement : 

To  develop  a  general  descriptive  model  of  the  cognitive  processes  of 
intelligence  analysis  and  to  discuss  how  specific  cognitive  skills  support 
the  performance  of  analytic  tasks. 

Procedure: 

The  research  approach  was  to  examine  the  role  and  activities  of 
var.ous  analysts  experienced  in  signal,  imagery,  and  all-source  intelligence 
processing  and  production.  In  addition  to  data  collected  through  inti 
views  and  observation,  the  research  literature  in  the  area  of  cognitive 
psychology  was  reviewed.  A  descriptive  model  of  the  cognitive  processes 
underlying  intelligen 'e  analysis  was  developed  based  on  the  general  prin¬ 
ciples  derived  from  the  literature  review  and  the  interviews  with  intelli¬ 
gence  personnel. 

Find ings: 

The  examination  of  intelligence  analysis  identitied  environmental  and 
individual  variables  as  well  as  underlying  cognitive  processes  which  con¬ 
tribute  to  the  quality  of  intelligence.  A  major  finding  was  that  intelli¬ 
gence  analysis  is  an  internal,  concept-driven  activity  rather  than  an  exter¬ 
nal,  data-driven  activity.  A  summary  of  the  early  findings  is  available  in 
ARI  Research  Report  1237.  The  present  report  builcts  on  the  findings  reported 
there  and  identify  ;  common  problems  associated  with  Iiuman  judgment  and  rcascnlng 
that  have  implications  for  the  training  and  support  requiranents  of  analysts. 

Utilization  of  Findings: 

Recently  the  research  findings  have  been  applied  to  the  development  of 
a  Training  Circular  titled  "An  Introduction  to  Tactical  Intelligence  Analysis: 
Cognitive  Preparation  for  the  Battlefield"  and  to  the  development  of  a  "Strate¬ 
gic  Intelligence  Analysis  Handbook"  (ITAC  Document  ATC-PP-2660-161-83 ) . 

Included  in  the  training  materials  are  an  overview  of  the  cognitive  skills 
of  analysis,  a  recommended  systematic  approach  to  performing  analysis,  and 
discussion  of  analytical  procedures  and  aids  to  support  analysis.  The 
research  findings  have  potential  application  to  analyst  performance  evalua¬ 
tion  and  to  the  identification  of  future  training  and  system  support  require¬ 
ments  . 
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1.  INTRODUCTION 


1.1  Purpose 

This  document  sorves  two  major  purposes:  (1)  Tc  provide  a  summary  of  the  IM1IA 
findings  and  (2)  to  identify  training  Issues  and  resoarch/information  gaps  Ir  the  cog¬ 
nitive  literature  rulevant  to  intelligence  analysis.  The  document  is  organized  into  nlnu 
chapters:  Following  the  Introduction,  Chapters  2  through  5  emphasize  theoretical 
aspects  of  the  background  material  used  to  develop  training  materials;  Chapters  G, 
7,  and  8  describe  relevant  issues  in  applying  the  theory  to  an  understanding  of  the 
day-to-day  activities  of  intelligence  analysts;  Chapter  9  discusses  research  and 
development  issues  and  training  Implications  derived  from  these  Issues  and  from  the 
theoretical  considerations  of  Chapters  2  through  5. 

IMTIA,  a  research  project  for  the  "Investigation  of  Methodologies  and  Techniques  for 
Intelligence  Analysis,"  was  sponsored  jointly  by  INSCOM  and  ARI.  IMTIA  research  cul¬ 
minated  in  the  development  of  a  generic  model  of  the  mental  processes  underlying 
Intelligence  ar  alysis.  The  theoretical  concepts  that  were  identified  and  defined  dur¬ 
ing  the  IMTIA  research  have  served  as  the  impetus  for  developing  training  materials 
for  strategic  and  tactical  analysts. 

Prior  to  the  IMTIA  research,  the  mental  processes  of  the  intelligence  analyst  were 
often  treated  as  if  they  were  in  a  "black  box",  i.e.,  Inaccessible  to  research.  There 
was  little  discussion  about,  and  even  less  training  to  improve,  these  mental 
processes.  IMTIA  represents  a  first  step  towards  describing  and  analyzing  the  cog¬ 
nitive  tasks  performed  by  analysts  within  the  context  of  the  analytic  production  sys¬ 
tem.  The  IMTIA  research  resulted  In  the  development  of: 

o  An  all-source  production  model. 

•  A  cognitive  model  of  the  intelligence  analyst. 

e  implications  of  the  model  for  training  and  evaluating  performance. 

The  first  eight  chapters  of  this  document  focus  on  he  following  five  areas  of  inquiry; 

1.  The  cognitive  underpinnings  of  Intelligence  analysis.  Chapter  2  describes  those 
cognitive  structures  and  processes  that  are  particularly  meaningful  for  under¬ 
standing  analytic  behavior,  and  hence,  for  developing  training  materials.  Chapter 
3  discusses  the  meaning  and  relevance  of  conceptual  models  and  shared  con¬ 
ceptual  models,  as  well  as  the  knowledge  requirements  underlying  optimum  ana¬ 
lytic  performance. 

2.  Generic  processes  and  associated  mental  tasks  that  pertain  to  all  analytic  perfor¬ 
mances.  Chapter  4  discusses  the  general  mental  skills  required  by  analysts. 
The  specific  mental  skills,  as  made  evident  In  nine  generic  task  segments  that 
underlie  all  analytic  tasks  are  described  in  Chapter  5.  These  task  segments  are 
mentally-oriented  performances,  that,  when  combined,  operate  to  fulfill  any  Intel¬ 
ligence  requirement. 

3.  The  threat  model  concept.  The  structure  and  processos  incorporated  within  the 
threat  model  are  described  in  Chapter  6.  The  threat  model  incorporates  the  ana¬ 
lytic  context,  products,  and  the  means  for  developing  those  products. 
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4.  Moans  for  evaluating  intelligence  training  and  pe>  formancc.  To  evaluate  ana¬ 
lytic  performance  and  training  needs,  the  concept  of  "ideaT'  states  (represent¬ 
ing  analyst,  performance,  and  product)  has  been  advanced. 

Evaluation  Is  the  process  of  compa’lng  hypothetical  Ideal  states  with  actual  per¬ 
formances  and  products.  The  Ideal  product  Is  arrived  at  by  developing  all 
aspects  of  the  threat  model  that  are  relevant  to  the  mission  requirements. 
Chapter  7  describes  the  concept  of  an  ideal  product  and  how  to  achieve  this 
ideal  product  by  following  the  steps  for  developing  the  threat  model. 

6.  Automated  data  processing  uses  and  potential  as  analytic  tools.  Chapter  8 
describes  existing  systems  ihat  support  the  analytic  process. 

The  above  areas  of  inquiry  have  served  to  identify  ways  to  Improve  analytic  perfor¬ 
mance  and  areas  that  can  be  enhanced  through  training. 

Chapter  9  identifies  training  Issues  and  research  and  information  gaps  in  the  cogni¬ 
tive  and  decision  making  literature  relevant  to  intelligence  analysis.  Identifying 
those  gaps  can  serve  as  a  basis  for  research  designed  to  advance  our  knowledge 
and  understanding  of  the  cognitive  basas  of  intelligence  analysis. 

This  document  can  also  serve  as  background  material  for  intelligence  analysts 
Interested  in  some  of  the  theoretical  concepts  underlying  cognitive  behavior  in  intel¬ 
ligence  analysis. 


1.2  Summary  of  IIVJTIA  Findings 

A  summary  view  of  the  analytical  process  and  its  context  is  shown  in  Figure  1-1. 

The  analyst  Is  shown  as  central  to  the  analytic  process,  and  as  Interacting  with,  and 
impacting  on,  the  intelligence  production  cycle  (users,  context,  requirements,  intelli¬ 
gence  product),  available  resources,  and  Work  setting  context. 

In  developing  a  model  to  represent  the  cognitive  activities  of  the  intelligence 
analyst,  LOGICON  combined  the  results  of  more  than  two  hundred  interviews  of 
analysts  on  the  job,  wit! i  extant  models  of  cognitive  thinking  and  of  problem-solving 
and  decision-making  behavior.  The  resultant  model  can  be  di-scribed  as  a  goal- 
oriented,  context-specific,  cognitive  model  of  intelligence  analysis. 

The  general  goal  for  any  Intelligence  analyst  is  to  reduce  the  uncertainties  of  the 
users  of  intelligence.  Reducing  uncertainties  involves  fulfilling  the  intelligence 
requirements  which  can  be  defined  in  terms  of  the  idea I  product. 

One  of  the  primary  aspects  of  intelligence  analysis  research  is  the  use  of  models  to 
represent  behavior,  environment,  and  communication.  The  use  of  such  models  has 
several  advantages: 

•  A  model  is  a  means  for  representing  only  those  aspects  of  a  concept  or  situation 
that  are  relevant  to  the  research. 

•  A  model  provides  a  way  to  define  a  subset  of  a  situation  of  interest. 

•  Models  force  one  to  identify  underlying  assumptions. 

•  Models  can  be  used  to  predict  behavior  or  situations,  and  hence,  to  anticipate 
potential  problems  in  performances  or  situations. 
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figure  1-1:  Model  of  the  Analytic  Process. 


•  Models  can  be  used  to  Identify  areas  of  uncertainty  with  respect  to  some 
hypothesis,  (e.g.,  in  the  world  situation,  for  training  research,  etc.) 

In  the  process  of  the  IMTIA  research,  several  different  types  of  models  were  used. 
The  major  models  were: 

•  The  cognitive  model  of  the  intelligence  analyst.  This  model  describes  the 
memory  structure  and  processes  of  the  cognitive  system,  emphasizing  those 
functions  that  are  relevant  to  intelligence  analysis. 

•  Conceptual  models  and  shared  conceptual  models.  A  conceptual  model  is  the 
functional  unit  in  memory  that  determines  how  humans  perceive  and  understand 
their  environment  and  how  they  communicate  with  each  other.  Shared  concep¬ 
tual  models  are  knowledge  areas  shared  by  two  or  more  individuals  (e.g., 
language,  mission  requirements,  etc.).  No  communication  is  possible  without 
shared  conceptual  models. 

•  The  cognitive  framework  of  the  intelligence  analyst.  This  is  an  expanded  version 
of  the  cognitive  model.  It  sets  the  cognitive  and  conceptual  models  of  the  intel¬ 
ligence  analyst  In  the  larger  framework  of  the  environment  in  which  the  analyst 
works.  This  framework  identifies  the  major  aspects  of  the  analyst’s  environment 
that  Impact  on  the  analyst’s  thinking  processes,  such  as  the  analyst’s  goals, 
mission  requirements,  user  requirements,  work  setting  variables,  analytic  tasks, 
and  Input  information  that  is  used  in  creating  the  analytic  product. 

•  The  production  model.  This  model  describes  the  intelligence  production  system 
as  It  interrelates  with  the  analytic  tasks  and  affects  the  operational  mission. 

•  The  threat  model.  This  is  a  model  of  the  geographic  environment,  and  of  the 
enemy  and  friendly  situations  within  which  the  analytic  product  Is  developed.  It 
represents  the  most  Important  conceptual  model  shared  by  analysts  and  the 
users  of  Intelligence. 

These  models  are  considered  separately  for  practical  purposes  and  for  clarification. 
In  reality,  the  phenomena  underlying  the  models  are  all  connected  and  the  aspects 
represented  by  each  model  interact  to  produce  a  comprehensive  view  of  the  working 
analyst  interacting  with  the  analytic  requirements. 

In  summary,  the  concepts  evolved  during  the  IMTIA  studies  and  represented  by  these 
models  can  be  used  to: 

—  Evaluate  analytical  performance  and  intelligence  products. 

—  Develop  training  materials  that  can  compensate  for  shortfalls  in  performance  and 
products. 

—  Improve  communications  effectiveness  throughout  the  production  cycle. 

—  Define  better  uses  of  automation. 

—  Devise  means  for  skill  maintenance  and  improved  transfer  of  training. 

The  major  findings  and  concepts  that  arose  from  the  IMTIA  study  are  summarized 
below: 

•  Intelligence  analysis  is  a  process  whereby: 
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—  Information  is  collected  In  response  to  stated  needs  and  requirements. 

—  Analysts  must  deal  with  problems  of  sparse  data  and  scarce  resources. 

—  Raw  Information  Is  transformed  (processed  and  analyzed)  so  as  to  answer 
specific  questions  concerning  a  real  or  potential  threat  to  national  Interests. 

—  The  transformed  Information  is  combined  Into  an  intelligence  product  and 
communicated  to  a  user. 

•  The  research  that  led  to  the  development  of  the  threat  model  Is  represented  by 
the  generic  cognitive  model  of  the  Intelligence  analyst.  The  model  emphasizes 
the  Importance  of  goals  as  the  Impetus  for  behavior. 

•  Goats  serve  as  the  basis  for  determining  "Idea!  states"  tor  analysts,  perfor¬ 
mance,  and  product.  Ideal  states  can  be  defined,  and  serve  as  a  basis  for 
evaluating  performance  and  products  of  Intelligence  analysis.  The  ideal  product 
serves  as  a  checklist  against  which  to  evaluate  an  actual  intelligence  product. 

•  If  the  Ideal  product  is  specified  In  sufficient  detail  for  a  particular  context  and 
user,  the  ideal  product  can  also  serve  as  a  basis  for  deterrrl"'ing  collection 
requirements  and  allocating  available  resources. 

•  Behavior  and  performance  are  only  meaningful  when  analyzed  with  respect  to 
context  (work  setting  variables  and  environment).  While  cognitive  processes 
and  cognitive  skills  are  incorporated  within  the  model  and  represent  generic 
processes  applicable  to  all  types  of  analysts,  specific  applications  of  the  model 
(e.g.,  to  training,  automation  Issues,  skill  maintenance,  etc.)  must  be  context- 
specific. 

•  There  are  Internal  and  external  contexts.  Internal  contexts  are  referred  to  as 
"conceptual  models".  Conceptual  models  and  shared  conceptual  models  are 
major  concepts  for  identifying  certain  performance  and  communications  issues. 
Conceptual  models  determine  hew  well  analytic  tasks  are  performed. 

•  Analytic  tasks  ere  performed  by  variously  comuining  nine  generic  task  segments. 
These  task  segments  represent  the  mental  performances  underlying  intelligence 
analysis.  The  cognitive  model  is  applicable  across  all  types  of  intelligence  dis¬ 
ciplines. 

•  External  contexts  are  described  „/  the  IMTIA  production  model  and  consist  of 
geographical  environments,  work  setting  variables,  other  analysts,  users,  data 
sources,  and  so  forth. 

•  The  construct  of  major  importance  to  an  intelligence  analyst  is  the  Threat  Model. 
The  threat  model  combines  internal  and  external  contexts  as  defined  above.  It 
Is  a  multi-dimensional  representation  of  the  battlefield  environment,  used  for 
Integrating  informational  elements,  analyzing  options,  externalizing  mental  con¬ 
cepts,  communicating  with  other  analysts  and  users,  and  making  predictions  of 
future  possibilities  and  potential  events. 

•  The  intelligence  product  itself,  as  wel!  as  changes  in  missions,  users,  require¬ 
ments,  and  environment,  serve  to  generate  new  collection  requirements  and  ner , 
analytical  questions. 
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2.  COGNITIVE  STRUCTURES  AND  PROCESSES 


This  chapter  provides  an  overview  model  of  the  generic  human  information  processing 
system  as  studied  in  cognitive  psychology,  with  particular  emphasis  on  features  that 
relate  to  intelligence  analysis.  This  model  represents  an  organizational  framework  for 
discussing  cognitive  structures  and  p  cesses,  but  does  not  depict  the  entire  cogni¬ 
tive  model  with  all  of  its  complexities  as  presented  throughout  this  report.  The  model 
consists  of  two  components:  static  cognitive  structures  and  dynamic  processes  that 
are  brought  to  bear  on  Information  held  in  the  system.  These  components  are 
hypothetical  in  nature  and  do  not  at  present  have  definite  physiological  correlates  in 
the  brain.  In  subsequent  sections,  the  information-processing  model  is  used  to  dis¬ 
cuss  several  performance  issues  relevant  to  effective  intelligence  analysis. 


2.1  A  General  Information-Processing  Model 

Models  of  human  information  processing  are  made  up  of  two  components:  (1) 
hypothetical  memory  structures  that  retain  information  and  (2)  processes  that 
operate  on  the  information  received  from  the  environment  and  that  direct  the  flow  of 
information  from  one  structure  to  another.  The  model  presented  in  Figure  2-1  sum¬ 
marizes  at  a  very  general  level  selected  aspects  of  cognitive  functioning  that  are 
logically  involved  in  intelligence  analysis. 

Depicted  on  the  left  side  of  the  figure  are  the  external  inputs  to  the  system  (the 
external  work  setting  context,  including  retrieval  of  old  data  from  Intelligence  jour¬ 
nals  or  other  external  memory  devices).  The  right-hand  section  of  the  figure 
represents  the  unobservable  processing  of  information  within  the  analyst’s  mind 
leading  to  observable  behavior.  This  processing  is  hypothesized  to  consist  of  the 
interaction  of  three  types  of  memory:  active  memory  (consciousness),  episodic 
memory,  and  semantic/factual  memory.  The  present  discussion  is  not  concerned  with 
lower-order  sensory  processes. 

Active  memory  contains  the  information  that  the  analyst  is  consciously  processing  at 
the  moment.  Active  memory  is  said  to  have  a  fixed  capacity  for  holding  and  operat¬ 
ing  on  information  (Baddeley  and  Hitch,  1974)  and  this  capacity  Is  believed  to  vary 
somewhat  from  individual  to  individual.  The  greater  the  capacity,  the  greater  the 
person’s  ability  to  aggregate  and  integrate  separate  elements  of  information  into 
higher-order  units.  It  is  believed  that  individuals  can  learn  "cognitive  economy"  or 
strategies  for  chunking  information  together  so  as  to  increase  the  total  amount  of 
information  that  can  be  held  in  active  memory  at  one  time,  but  the  number  of  chunks 
that  can  be  held  at  one  time  (the  capacity  of  active  memory)  cannot  be  increased 
with  training  (Chase,  1978).  The  use  of  learned  conceptual  models  of  the  world  as 
Information  chunking  devices  (see  Chapter  3)  provides  one  method  for  circumventing 
the  limited  capacity  of  active  memory.  For  example,  a  large  amount  of  data  concern¬ 
ing  a  certain  configuration  of  enemy  units  and  movement  could  be  summarized  simply 
83  a  doctrinal  attack  pattern.  The  use  of  automated  memory  aids  and/or  team 
memory  provides  an  additional  strategy  for  reducing  the  likelihood  of  information 
overload  In  active  memory  (see  Sections  2,5.1  and  2.5.2).  The  more  complex  the 
operations  being  performed  in  active  memory,  the  smaller  the  amount  of  information 
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that  can  be  operated  on  at  that  time  (Geiselman  and  Bellezza,  1975).  For  example, 
rote  Journalizing  of  data  would  be  less  likely  to  interfere  with  the  ongoing  monitoring 
of  Incoming  information  than  would  hypothesis  generation. 

Episodic  memory  contains  a  record  of  the  perceptions  and  thoughts  (encoding) 
experienced  by  the  analyst.  The  greater  the  frequency  of  processing  and/or  the 
amount  of  processing  given  to  information,  the  greater  the  consolidation  of  that  infor¬ 
mation  In  memory.  The  consolidation  process  is  discussed  in  Section  2.2.2.  The 
episodic  record  Is  ordered  temporally  and  is  believed  to  have  no  identifiable  capacity 
limitations.  Although  some  memory  theorists  speculate  that  no  information  is  ever 
erased  from  episodic  memory,  all  agree  that  some  memories  are  more  accessible  than 
others.  Memory  accessibility  varies  from  time  to  time  and  with  the  immediate 
retrieval  context.  Although  recognition  memory  is  typically  higher  than  recall  memory, 
manipulation  of  the  context  at  the  time  of  memory  retrieval  can  produce  situations 
where  recall  performance  actually  exceeds  recognition  performance  (Flexser  and 
Tulvlng,  1978).  It  Is  important  for  the  intelligence  analyst  to  recognize  memory- 
retrieval  shortfalls  so  that  external  memory  aids  are  utilized  effectively  and  memory 
inaccessibility  or  non-recognizability  is  not  Interpreted  as  an  absence  of  the  informa¬ 
tion  in  question.  Problems  of  memory  access  relevant  to  intelligence  analysis  are 
discussed  in  Section  2.2.3. 

Semantic  and  factual  memory  contains  a  knowledcie  base  that  is  not  temporally  dated 
or  linked  to  specific  episodes  (such  as  definitions  of  terms  and  doctrine)  or  to 
organizational/procedural  structures  (rules  of  logic,  conceptual  models,  prototypes, 
goals,  plans,  biases,  skill  routines).  Information  structures  held  in  semantic  and  fac¬ 
tual  memory  provide  the  basis  for  interpreting  the  world.  The  development  of 
"expert  systems"  is,  in  part,  directed  toward  modeling  and  understanding  the 
semantic/factual  knowledge  of  individuals  who  show  a  high  levei  of  performance 
within  a  particular  domain  of  Interest.  A  major  goal  of  the  human  information¬ 
processing  system  is  to  make  Irregular  patterns  of  information  regular.  This  goal  is 
reflected  In  the  Interaction  between  semantic/factual  memory  and  active  memory  as 
depicted  in  Figure  2-1 .  Assimilation  refers  to  the  modification  and  elaboration  of  new 
Information  to  fit  prior  conceptions  or  hypotheses  (i.e.,  interpretation  of  information 
through  selection  and  c  "ineralization,  see  Section  2.2.1).  Accomodation  refers  to  the 
modification  of  the  existing  contents  of  memory  (e.g.,  within  a  conceptual  model)  to 
accept  new  or  inconsistent  information.  These  two  processes  are  the  mechanisms 
through  which  knowledge  is  acquired.  When  assimilation  is  carried  to  an  extreme,  a 
bias  toward  confirming  pre-conceived  hypotheses  is  in  evidence  (as  with  a  confirma¬ 
tion  bias,  see  Section  2.4.4);  whereas  when  accomodation  is  carried  to  an  extreme, 
the  analyst  may  disregard  the  probabilistic  nature  of  intelligence  data  and  exhibit  a 
bias  toward  switching  hypotheses  upon  receipt  of  minimally  conflicting  information. 
Both  of  these  extreme  tendencies  are  more  likely  to  occur  under  conditions  of 
stress. 

The  analyst  can  avoid  the  possible  negative  consequences  of  the  extreme  cases  of 
assimilation  and  accomodation  by  utilizing  an  evaluation  structure  that  allows  weight¬ 
ing  of  evidence  in  both  past  and  future  episodes  of  the  ongoing  scenario.  Over- 
assimilation  can  be  avoided  by  retaining  a  structure  for  indicators  and  episodes  of 
alternative  hypotheses.  Over-accomodation  can  be  avoided  by  forcing  an  archival 
justification  of  new  hypotheses  to  the  equivalent  level  of  established  hypotheses. 
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2,2  Information  Contents  Modification  Cycle 

Cognition  within  the  above  information-processing  system  may  be  characterized  as  a 
set  of  interrelated  processes  that  operate  on  the  available  information  contents  in 
memory  and  modify  them. 

Since  information  from  the  senses  and  in  memory  constitutes  the  raw  material  upon 
which  intelligence  analysis  is  based,  Information  modification  mechanisms  have  impor¬ 
tant  Implications  for  understanding  and  predicting  the  orientations  and  nature  of  ana¬ 
lytic  interpretations  and  estimates.  The  descriptive  model  of  analytic  behavior 
developed  here  builds  on  an  understanding  of  basic  cognitive  processes  and 
attempts  to  explain  analysts'  Interpretation,  storage,  and  recall  of  information.  At  a 
general  level,  the  model  describes  the  dynamic  interplay  between  incoming  informa¬ 
tion  and  previously  stored  information  (i.e.,  internal  memories).  Processes  that  are 
central  to  this  interplay  encompass  the  memory  modification  cycle  involving  informa¬ 
tion  filtering,  memory  consolidation,  and  memory  access  interference. 

Associated  with  these  mechanisms  ate  several  shortfalls  in  information  processing 
tied  to  limitations  of  the  human  information  processing  system.  These  problems  are 
presented  and  candidate  safeguard.,  or  solutions  are  offered.  1  some  cases,  simple 
awareness  of  the  problems  may  limit  their  occurrence. 


2.2.1  INFORMATION  FILTERING 

The  Information  filtering  mechanism  is  composed  of  two  complementary  functions: 
selectivity  and  generalization. 

The  selectivity  mechanism  filters  the  raw  information  pattern  and  selects  out  those 
aspects  that  are  significant.  This  is  done  by  comparing  the  contents  of  the  currently 
active  conceptual  model  (see  Chapter  3)  with  the  raw  information.  If  an  adequate 
match  is  found  that  tags  the  raw  Input  as  significant,  the  input  is  assimilated  into  the 
analyst’s  data  base  as  a  member  of  an  existing  mental  category. 

This  initial  comparison  often  results  in  passively  rejecting  significant  information  in 
the  raw  input  because  it  does  not  fit  the  mental  category  assigned  to  it.  It  the 
overall  first  impression  of  the  input  information  pattern  is  a  good  match  with  the 
gross  features  of  existing  memory  contents,  disparities  between  the  input  pattern 
and  the  memory  Information  pattern  at  more  detailed  levels  are  frequently  not  even 
noticed.  Thus,  actual  disparities  are  ignored  and  the  erroneously  perceived  informa¬ 
tion  is  assimilated  into  the  existing  conceptual  model  (see  Figure  2-1).  Selectivity 
reduces  memory  load  for  specific  instances,  but  details  that  are  ignored  may  later 
turn  out  to  be  significant. 

Selectivity  is  biased  by  expectations.  This  bias  is  the  result  of  a  mechanism  called 
polarization  filtering.  Polarization  filtering  is  a  variation  of  selectivity  filtering,  in 
which  an  expectation  that  has  been  established  increases  the  accessibility  of 
memory  contents  related  to  that  expectation.  This  includes  information  for  confirming 
or  denying  ext  ctations,  although  In  general,  positive  expectations  are  more  pre¬ 
valent,  leading  to  the  conflrmat’  .  bias  (see  Section  2.4.4).  When  these  expecta¬ 
tions  are  related  to  formal  analytic  hypotheses,  polarization  filtering  leads  to  accept¬ 
ing  such  hypotheses  with  insufficient  evidence  (Type  1  error).  The  polarization 
effect  focuses  attention  on  the  features  of  the  expectation,  thus  passively 
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rejecting  other  potentially  important  Information  that  happens  to  be  irrelevant  for 
confirming  or  denying  the  expectation. 

The  polarization  effect  can  bring  positive  results  when  unfolding  events  correspond 
to  expectations,  and  negative  results  when  events  are  unrelated  to  confirming  or 
denying  expectations.  Polar'zing  effects  are  stronger  when  an  expectation  is  impli¬ 
cit  (l.e.,  Is  unexamined  or  unquestioned  in  awareness).  Therefore,  explicit  question¬ 
ing  of  expectations  and  identification  of  underlying  assumptions  can  reduce  polariza¬ 
tion.  This  is  the  reason  that  the  IMTIA  research  has  identified  the  need  for  intelli¬ 
gence  analysts  to  develop  their  skills  in  "identifying  assumptions"  (see  "Handbook 
for  Strategic  Intelligence  Analysis"). 

in  contrast  to  selectivity  mechanisms,  generalization  mechanisms  filter  raw  input 
information  by  determining  the  types  and  degrees  of  similarities  required  to  recognize 
things  as  members  of  well-known  catagories.  The  confident  use  of  acquired 
knowledge  depends  on  being  able  to  generalize  from  experience.  Generalization  is  a 
fundamental  process  for  organizing  large  numbers  of  unique  instances  into  manage¬ 
able  form.  Success  in  applying  past  experience  (memory  information)  to  the  present 
depends  on  the  validity  of  the  generalizations  employed.  Given  that  generalization 
often  ignores  significant  differences  between  specific  instances,  such  differences 
should  be  recorded  as  appended  information  in  the  event  that  the  generalizations 
should  later  prove  invalid. 


2.2.2  MEMORY  CONTENTS  CONSOUDATION 

Memory  contents,  including  information  recently  passed  through  the  filtering  process 
and  stored  in  episodic  memory,  are  consolidated  (l.e.,  made  more  accessible  and 
vivid)  as  a  joint  function  of  the  frequency  of  processing  and  the  amount  of  attention 
used  In  the  processing.  Thus,  more  frequently  encountered,  important  types  of 
experiences  upon  which  significant  mental  effort  is  expended  become  more  vivid  and 
immediately  accessible  in  memory. 

The  increased  accessibility  and  vividness  of  particular  memory  contents  increases 
the  likelihood  that  they  will  be  used  as  filtering  criteria  for  future,  somewhat  similar 
raw  experiences.  For  this  reason,  the  contents  consolidation  mechanism  can  have 
important  implications  for  the  accuracy  of  analytic  interpretations  and  estimates.  If 
the  results  of  the  consolidation  mechanism  match  the  realities  of  future  events  to  be 
interpreted,  the  effects  of  consolidation  are  advantageous;  if  not,  the  effects  are 
detrimental.  Long  term  static  conditions  tend  to  increase  the  positive  aspects  of  the 
consolidation  process,  while  eras  of  rapid  and  significant  change  do  not. 

Consolidation  can  result  in  a  phenomenon  called  the  caricature  effect,  a  type  of  dis¬ 
tortion  of  the  input  information  following  mental  rehearsal  of  an  experience,  rumina¬ 
tion  about  an  experience,  or  problem-solving  behavior  about  an  experience.  These 
cognitive  activities  can  increase  the  accessibility  and  the  vividness  of  the  particular 
memory  contents  related  to  that  experience.  Given  no  additional  external  informa¬ 
tion  about  a  certain  experience,  continued  rehearsal,  rumination,  and  thought  tend  to 
emphasize  and  de-emphasize  various  aspects  of  the  memory  of  that  experience. 

The  result  of  emphasis  and  de-emphasis  is  to  "normalize"  usual  or  expected  aspects 
of  the  memory  and  to  exaggerate  unusual  or  unexpected  aspects,  with 
usualness/unusualness  being  judged  in  relation  to  the  rest  of  the  overall  memory 
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structure.  That  Is,  the  consistency  of  usualness  between  some  of  the  contents  of 
the  particular  memory  and  the  balance  of  memory  contents  may  be  exaggerated 
beyond  their  original  consistency,  and  the  disagreement  and  inconsistency  in  other 
parts  of  that  particular  memory  may  also  be  exaggerated  beyond  their  original  condi¬ 
tion. 

Since  the  combined  results  of  these  processes  tend  to  produce  a  memory  that  is  a 
caricature  of  the  original  contents,  the  result  is  termed  the  caricature  effect.  This 
effect  tends  to  feed  on  the  elements  of  unusualness  and  surprise  and  to  overem¬ 
phasize  these  elements  as  compared  to  the  more  expected  elements  of  the  experi¬ 
ence.  If  the  novel  elements  of  an  experience  are  accurate  components  of  a  future 
similar  event,  the  caricature  effect  may  provide  help  in  interpreting  the  future  event. 
If  not,  the  caricature  effect  can  impede  accurate  interpretation,  especially  if  the 
Interpretation  must  be  based  on  incomplete  data. 

The  cariceture  effect  is  a  special  "no  new  information"  version  of  the  consolidation 
mechanism  (the  latter  being  based  on  repeated  instances  of  a  certain  pattern  of 
external  experience).  Since  the  caricature  effect  depends  partly  on  the  experience 
of  initial  surprise  followed  by  unshared  and  unexamlned  rehearsal  and  rumination,  the 
conditions  for  predicting  and  controlling  the  caricature  effect  are  at  present  only 
partially  understood.  This  effect  might  be  minimized  by  reviewing  the  journalized 
record  of  the  initial  interpretation  and  perhaps  by  discussing  the  unusual  aspects 
with  other  analysts. 


2.2.3  MEMORY  ACCESS  INTERFERENCE 

For  memories  to  be  useful  to  the  analyst,  the  analyst  must  be  able  to  access  them. 
Accessing  a  memory  of  an  earlier  event  occurs  in  one  of  two  ways:  recall  or  recogni¬ 
tion. 

•  Recall  consists  of  accessing  the  memory  contents  from  an  earlier  experience 
when  receiving  a  name  or  description  of  the  situation  within  which  that  event 
was  experienced.  Recall  consists  of,  for  example,  responding  to  the  question 
"What  kinds  of  vehicles  were  present  in  the  imagery  you  viewed  before  lunch 
yesterday?" 

•  Recognition  consists  of  accessing  a  memory  for  an  earlier  situation  that  matches 
currently  presented  specific  information.  Recognition  consists  of,  for  example, 
responding  to  the  question  "Is  this  frame  of  imagery  the  same  as  one  that  you 
viewed  before  lunch  yesterday?" 

Interference  In  accessing  memory  occurs  for  both  recognition  and  recall.  Memory 
retrieval  is  most  efficient  when  the  memories  are  discriminable.  Memories  for  very 
similar  experiences  can  interfere  with  one  another  during  memory  access  from 
episodic  memory,  slowing  access  and  making  it  less  reliable  and  less  accurate.  Such 
Interference  can  have  strong  effects  on  the  memory  information  available  for  the 
filtering  stage  of  the  next  cycle. 

The  two  main  interference  effects  are  the  Intervening  similarities  effect  and  the 
similarities  saturation  effect. 

For  both  recall  and  recognition,  highly  similar  experiences  that  have  intervened 
between  the  original  experience  and  the  current  requirement  for  memory  access 
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tend  to  Interfere  with  the  accessibility  of  the  original  memory  material;  the  interven¬ 
ing  similarities  effect  creates  interference  with  memory  access  for  both  recall  and 
recognition.  Thus,  an  analyst  processing  many  messages  of  very  similar  contents 
from  the  same  domain,  under  constant  conditions,  and  over  an  extended  period  of 
time,  Is  unlikely  to  be  able  to  recall  the  specific  messages  processed  during  a  cer¬ 
tain  period  of  time.  Also,  the  analyst  may  not  be  able  to  recognize  a  specific  mes¬ 
sage  presented  for  re-examlnatlon  as  having  ever  been  processed. 

External  memory  aids  should  be  used  when  Intervening  similarities  interference  can 
be  anticipated,  and  when  it  must  be  circumvented.  The  use  of  external  memory  aids 
is  discussed  in  Section  2.5.1.  When  interference  cannot  be  anticipated,  and  is 
recognized  only  after  the  events  have  occurred,  the  analyst  should  try  to  mentally 
reinstate  the  context  surr-  ding  the  event  in  question,  drawing  upon  any  unusual 
details  that  might  make  the  i..amories  discriminable. 

The  similarities  saturation  effect  occurs  following  concentrated  repetitions  of  highly 
similar  expediences  that  saturate  related  areas  of  memory  with  many  highly  similar 
memory  contents.  This  increases  the  difficulty  of  comparing  across,  and  discriminat¬ 
ing  between,  many  similar  memories,  and  causes  reduced  speed  and  accuracy  in  the 
processing  of  each  new  related  experience.  It  also  Interferes  with  rapid  and  discri¬ 
minable  storage  of  the  similar  new  experiences  in  memory.  The  similarities  saturation 
effect  cen  be  lessened  by  providing  the  individual  a  chance  to  refocus  attention  on 
different  memory  contents,  if  possible,  thus  allowing  the  interfering  memories  to 
become  less  vivid  and  less  immediately  accessible.  Following  recovery  from  satura¬ 
tion,  the  capacity  for  new  discriminations  In  that  area  of  memory  is  restored. 

The  intervening  similarities  and  similarities  saturation  forms  of  interference  with 
memory  performance  are  predictable  cognitive  mechanisms  of  Information  processing. 
They  operate  to  weaken  and  diffuse  the  experiential  Information  available  from 
episodic  memory  by  affecting  the  speed,  reliability,  and  accuracy  of  access  to 
memory  contents.  Such  weakening  and  diffusion  can  change  the  pattern  of  the 
memory  contents  that  will  be  used  as  filtering  criteria  for  the  next  cycle  of  experi¬ 
ence  and  memory  modification. 

In  summary,  there  are  throe  potentially  predictable  and  controllable  cognitive 
mechanisms  that  operate  in  a  cycle  to  modify  information  contents  available  from 
memory.  Since  memory  contents  provide  a  large  portion  of  the  information  used  in 
making  many  intelligence  analysis  interpretations  and  estimates,  the  information  con¬ 
tents  modification  cycle  is  an  important  concept  for  suggesting  ways  to  improve 
intelligence  analysis. 

Within  this  cycle,  information  is  filtered,  consolidated,  and  otherwise  modified. 
Selective  filtering  may  operate  to  ignore  (filter  out)  aspects  of  the  input  information 
that  diverge  from  stored  information.  Polarization,  stemming  from  established  expec¬ 
tations,  may  increase  the  chance  of  processing  information  that  would  otherwise 
have  been  filtered,  but  it  may  also  lead  to  filtering  of  other  information  not  directly 
related  to  confirmation  or  denial  of  the  expectancy.  Generalization  is  an  important 
mechanism  that  operates  during  the  filtering  process  to  aggregate  large  amounts  of 
data  into  manageable  form. 

!r>'ut  Infoimaifon  that  has  passed  though  the  fllte  ing  process  is  consolidated  with 
pre  exist!; ij  information  contents.  Tn^  oonsoiklatu  n  process  Increases  access  to 
frequently  used  informabon,  but  it  may  also  lead  to  various  distortions  of  the 
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Information.  Accessibility  to  memory  contents  is  also  determined  by  the  relationships 
of  various  kinds  of  Information  In  storage.  For  example,  input  Information  that  is 
highly  similar  to  stored  Information  can  create  confusion  and  interfere  with  recall  or 
recognition. 

2.2.4  AWARENESS  OF  MEMORY  FUNCTION 

Often  we  are  not  aware  that  we  use  Information  that  comes  from  memory.  A  simple 
Illustration  of  this  phenomenon  can  be  seen  in  the  contributions  of  memory  to  the 
task  of  Identifying  a  military  vehicle  masked  by  a  tree.  Visible  parts  of  the  vehicle 
provide  the  cues  for  matching  and  decoding  memory  contents  and  reconsti  ucting  the 
visually  missing  parts  of  the  vehicle.  As  the  fill-in  is  accomplished,  the  image  of  the 
tree  Is  effectively  dimmed  or  the  event  erased  from  consideration.  At  this  point  the 
tree  Is  down-graded  or  eliminated  from  awareness,  and  a  "camouflaged  tank  retri¬ 
ever"  Is  confidently  reported.  The  process  of  using  Information  from  memory  for  fill- 
in  is  usually  dismissed  or  not  even  noticed.  The  same  sequence  of  data  occlusion, 
fill— in,  and  downgrading  of  irrelevant  information  occurs  continuously  for  conceptually 
more  complex  and  subtle  forms  of  experiences  associated  with  analysis.  Filt-in  is  a 
useful  process  because  it  allows  for  Interpretation  and  prediction  when  only  partial 
information  is  available,  but  it  can  also  lead  to  premature  Interpretations  of  the  data. 
Data  occlusion  and  downgrading  of  information  are  also  useful  and  Indispensible  infor¬ 
mation  processing  mechanisms,  used  for  organizing  and  filtering  data;  they  save  time 
but  they  can  also  lead  to  inadvertent  oversights  of  potentially  Important  information. 
Intelligence  analysts  must  be  aware  of  these  limits  of  the  cognitive  mechanism. 


2.3  Positive  and  Negative  Aspects  of  the  Processing  System 

The  structural  and  processing  characteristics  just  described  have  both  positive  and 
negative  aspects  associated  with  It.  The  constraints  on  the  human  Information  pro¬ 
cessing  system  have  the  following  positive  results: 

e  Constraints  make  it  possible  to  organize  multitudes  of  environmental  stimuli  into 
meaningful  categories. 

•  Without  categorization,  meaning  could  not  be  assigned  to  the  various  perceptual 
inputs. 

•  Perceptual  inputs  could  not  be  as  gned  relevance  values. 

•  Similarities  between  stimuli  that  lead  to  the  assignment  of  items  to  categories 
would  not  be  recognized. 

•  No  reference  points  wouid  be  available  to  make  judgments  or  predictions. 

•  Patterns  would  not  be  recognized  as  meaningful. 

•  Probability  assignments  of  future  possibilities  would  be  impossible. 

In  other  words,  characteristics  such  as  filtering,  assimilation,  and  consolidation  are 
adaptive  processes.  They  can  be  thought  of  as  rules  Imposed  by  the  cognitive  sys¬ 
tem  that  are  responsible  for  organizing  environmental  stimuli  Into  meaningful  informa  ¬ 
tion  and  make  It  possible  for  humans  to  deal  with  new  and  with  old  information.  At 
the  same  time  they  also  tend  to  distort  the  true  picture  of  the  world  and  in  that 
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capacity,  they  are  at  the  basis  of  cognitive  biases.  Cognitive  biases  have  received 
a  great  deal  of  attention  in  the  Information  processing  literature  and  are  particularly 
relevant  to  intelligence  analysts,  who  must  make  inferences  based  on  uncertain  and 
sparst.  data.  While  the  rules  of  the  cognitive  system  tend  to  work  well  In  everyday 
altuations,  Intelligence  analysts  deal  with  situations  that  are  much  more  structured 
and  that  require  more  discipline.  In  such  situations,  the  affects  of  cognitive  biases 
tend  to  be  more  pronounced  and  consequential.  These  effects  can  be  partially  cir¬ 
cumvented  through  awareness. 

In  the  following  paragraphs,  the  major  recognized  biases  are  briefly  described. 


2.4  Cognitive  Biases 

The  term  bias  refers  to  a  subjective  point  of  view.  Typically,  the  term  is  used  to 
indicate  preconceived  (and  generally  false)  notions,  attitudes,  or  judgments  about 
something  or  someone.  Biases  are  the  result  of  being  associated  with  specific 
environments,  or  they  are  the  result  of  specific  characteristics  associated  with  the 
human  Infor  nation  processing  system.  Biases  can  be  categorized  as  fo'lows: 

•  Cultural  and  Personal. 

•  Organizational. 

•  Cognitive. 

Cultural  biases  are  constraints  on  ones  thinking,  acquired  during  maturation  from 
widely  held  beliefs,  practices,  or  cognitive  styles  that  characterize  one’s  specific 
social  environment.  Personal  biases  are  constraints  that  arise  from  specific  past 
experiences  of  the  Individual.  Organizational  biases  are  constraints  on  cognitive 
flexibility  imposed  by  local  Information,  goats,  mores,  and  traditions,  that  have 
evolved  within  the  specific  organization  in  which  the  individual  serves.  In  many 
Instances,  cultural,  personal,  and  organizational  biases  are  In  tact  identical  to  the 
underlying  assumptions  that  were  discussed  in  the  previous  section. 

Cognitive  biases  differ  from  the  above  in  that  they  are  to  a  large  extent  inherent 
characteristics  of  the  way  humans  think,  both  in  the  way  they  recall  information  from 
memory  and  In  the  way  they  process  (perceive  and  understand)  information  from 
their  environment. 

All  humans  are  influenced  by  biases.  The  important  issue,  for  analysts,  is  to  recog¬ 
nize  the  types  of  biases  that  exist  and  be  aware  of  the  potential  influences  that 
these  biases  may  have  on  intelligence  analysis. 

While  cultural,  personal,  and  organizational  biases  tend  to  distort  one’s  view  of  the 
world,  cognitive  biases  are  not  necessarily  detrimental  to  one’s  thinking.  In  the 
absence  of  Information,  a  preconceived  idea  ubout  something  can  at  least  give  the 
analyst  a  starting  point  for  thinking  about  a  situation.  However,  It  is  critical  for  the 
analyst  to  realize  that  the  source  of  the  idea  is  internal,  and  that  the  uncertainty 
level  associated  with  it  is  quite  high. 

In  general,  cognitive  biases  tend  to  distort  what  is  remembered,  how  It  is  remem¬ 
bered,  as  well  as  how  information  Is  evaluated.  Several  of  the  more  common  cogni¬ 
tive  biases  are  discussed  below. 
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2.4.1  SELECTIVITY  BIAS 


Information  Is  selectively  recalled  as  a  function  of  how  vivid,  concrete,  and  personal 
It  Is.  Vivid  Information  has  a  greater  Impact  on  thinking  than  pallid,  abstract  informa¬ 
tion  that  may  objectively  have  greater  value  as  evidence.  Information  that  is  per¬ 
sonally  perceived  Is  also  likely  to  be  better  remembered  than  information  received 
secondhand.  Initial  impressions  and  Items  that  are  first  in  a  series  also  tend  to  be 
more  vivid,  and  hence,  better  remembered. 

Intelligence  analysts  generally  work  with  secondhand  information.  On  occasions 
when  the  analyst  directly  perceives  information,  such  as  during  foreign  travel  or 
through  direct  communication  with  a  national  from  a  particular  court. ry,  these  events 
and  Information  will  become  especially  noteworthy.  Such  vivid  experiences  are  often 
a  source  of  new  Insights,  but  they  can  also  be  a  cause  of  self-deception,  and  hence, 
they  can  bias  your  interpretation  of  a  given  situation.  In  the  instance  of  foreign 
travel,  the  visitor  tvpically  will  become  familiar  with  only  a  small  sample  of  people 
representing  a  narrow  segment  of  the  total  society.  Incomplete  and  distorted  per¬ 
ceptions  are  a  common  result  of  the  selectivity  bias. 


2.4.2  AVAILABILITY  BIAS 

The  ability  to  recall  instances  'f  an  event  is  influenced  by  how  recently  an  event 
occurred,  by  personal  involve  int,  by  how  important  it  seemed  at  the  time,  and  by 
vivid  details.  All  of  these  factors  are  unrelated  to  the  true  probability  of  an  event. 
These  factors  do,  however,  Influence  our  judgment  by  making  recall  of  such  events 
more  easily  "available"  from  our  memory. 

When  making  judgments  about  the  likelihood  or  frequency  of  certain  events,  the 
availability  rule  of  thumb  Is  used.  According  to  this  rule,  the  probability  of  some 
event  is  judged  by  the  ease  of  imagining  .elevant  instances  of  that  event  or  the 
number  of  such  events  th*(t  we  can  easily  remember.  The  availability  rule  often 
works  quite  well,  but  it  can  be  misleading  when  the  recalled  vividness  of  an  event  is 
unrelated  to  Its  probability. 

Using  the  availability  rule  is  a  time  saver,  but  the  intelligence  analyst  must  be  aware 
of  such  shortcuts  and  recognize  the  strengths  and  weaknesses  of  the>  use. 


2.4.3  ABSENCE  OF  EVIDENCE  BIAS 

A  principal  characteristic  of  intelligence  analysis  is  that  key  information  is  generally 
lacking.  Analytical  problems  are  selected  on  the  basis  of  their  importance  and  the 
perceived  needs  of  the  users,  without  much  regard  for  availability  of  information. 
Analysts  must  do  the  best  they  can  with  limited  information,  but  they  must  also  anti¬ 
cipate  the  gaps  and  somehow  take  into  account  the  fact  that  relevint  information  is 
known  to  be  missing.  Missing  data  are  a  normal  characteristic  of  intelligence  prob¬ 
lems.  Research  has  shown  the  difficulty  that  even  experts  have  in  recognizing  and 
Incorporating  missing  data  Into  judgments  of  abstract  problems. 

The  notion  "out  of  sight,  out  of  mind"  should  not  be  a  description  of  the  impact  of 
gaps  In  Information.  The  analyst  needs  to  be  able  to  explicitly  identify  those 
relevant  variables  on  which  Information  Is  lacking,  consider  alternative  hypotheses 
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concerning  the  true  status  of  those  variables,  then  modify  their  judgment  (and  espe¬ 
cially  their  level  of  certainty)  accordingly.  It  is  also  relevant  to  consider  whether  a 
lack  of  information  on  such  variables  is  normal,  or  whether  the  absence  of  information 
Is  itself  an  indicator  of  unusual  activity  or  inactivity. 


2.4.4  CONFIRMATION  BIAS 

The  confirmation  bias  is  the  result  of  the  tendency  to  perceive  events  in  such  a  way 
as  to  confirm  existing  beliefs.  It  can  occur  in  one  of  two  ways.  The  first  occurs 
because  of  a  tendency  to  only  perceive  events  that  fit  within  existing  conceptual 
models.  The  second  way  is  distorting  the  meaning  of  what  is  seen,  so  that  it  fits 
preconceived  ideas. 

The  confirmation  bias  is  very  pervasive;  it  is  a  result  of  a  need  to  understand  the 
environment  in  terms  of  what  Is  already  know.  This  need  leads  to  perceiving  what  is 
expected  to  be  perceived. 

This  Is  important  to  remember  during  the  process  of  generating  hypotheses  about  a 
situation  or  some  future  event.  The  confirmation  bias  causes  the  perception  or 
interpretation  of  information  in  a  way  that  will  confirm  hypotheses  that  already  exist. 
At  the  same  time,  this  biase  can  prevent  the  realization  that  the  new  data  do  not 
support  the  existing  hypotheses. 


2.4.5  OVER-CONFIDENCE  BIAS 

A  large  component  of  any  analyst's  job  is  to  summarize  complex  ensembles  of  infor¬ 
mation  Into  dichotomous  judgments.  For  instance,  an  analyst  might  have  to  decide 
whether  a  particular  set  of  maneuvers  are  exercises  or  the  early  stages  of  an 
attack.  Or,  on  the  basis  of  personal  impressions  and  reports,  an  analyst  might  have 
to  decide  whether  a  particular  informant  is  or  is  not  competent. 

An  important  aspect  of  such  judgment  tasks  is  the  degree  of  confidence  that  accom¬ 
panies  them.  That  confidence  may  determine  whether  or  not  more  information  will  be 
gathered,  or  whether  an  action  will  be  taken. 

In  general,  there  is  a  tendency  to  be  overly  confident  in  their  ability  to  make  those 
types  of  Judgments.  Even  with  minimal  information  about  a  topic,  there  is  a  tendency 
to  generate  a  great  number  of  hypotheses  concerning  a  judgment  task  without  test¬ 
ing  these  hypotheses  properly.  Over-confidence  in  judgments  has  been  found  to  be 
the  rule,  rather  than  the  exception.  Such  over-confidence  may  lead  to  premature 
cessation  of  information  gathering  and  to  ineffective  decision  making  The  most 
effective  way  to  overcome  this  type  of  bias  is  to  be  aware  of  it. 


2.4.6  THE  OVER-SENSITIVITY  TO  CONSISTENCY  BIAS 

Internal  consistency  in  a  pattern  of  evidence  is  a  major  determinant  of  confidence  In 
Judgments  based  on  that  evidence.  In  one  sense,  consistency  Is  an  appropriate 
guideline  for  evaluating  evidence.  Alternative  explanations  or  estimates  are  formu¬ 
lated  and  one  selected  that  encompasses  the  greatest  amount  of  evidence  within  a 
logically  consistent  scenario. 
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Under  some  circumstances,  however,  consistency  can  be  deceptive.  Information  may 
be  consistent  only  because  it  is  redundant,  in  which  case  many  related  reports  may 
be  no  more  Informative  than  a  single  one  of  them.  Or  it  may  be  consistent  only 
because  the  information  came  from  a  very  small  sample  or  a  biased  sample. 

When  working  with  a  small  but  consistent  body  of  evidence,  analysts  need  to  con¬ 
sider  how  representative  that  evidence  is  of  the  total  body  of  potentially  available 
Information.  If  the  analyst  is  stuck  with  only  a  small  amount  of  evidence  and  cannot 
determine  how  representative  the  evidence  is,  confidence  in  judgments  based  on 
this  evidence  should  be  low  regardless  of  the  consistency  of  the  information. 


2.4.7  THE  RELIABILITY  BIAS 

There  is  a  tendency  to  deal  with  Information  at  face  value,  regardless  of  the  reliabil¬ 
ity  of  that  information.  There  are  many  reasons  why  Information  may  be  less  than 
perfectly  reliable:  small  sample  size  that  is  not  representative  of  the  totality  of  the 
Information;  misperception  or  bias  on  the  part  of  the  source;  distortion  in  the  report¬ 
ing  chain;  misunderstanding  or  misperception  on  the  part  of  the  analyst.  Further, 
some  of  the  Information  used  In  analysis  is  retrieved  from  the  analyst’s  memory,  and 
the  degree  of  reliability  originally  attributed  to  the  information  may  have  been  long 
forgotten. 

Analysts  generally  must  consider  many  items  of  information  with  different  degrees  of 
reilabF  ’  that  are  related  in  complex  ways.  It  is  unlikely  that  the  analyst  can  make 
neat  mathematical  or  even  intuitive  calculations  that  take  all  reliability  factors  into 
account.  There  seems  little  the  analyst  can  do  about  this  problem  short  of  breaking 
the  problem  down  In  a  way  that  permits  assigning  probabilities  to  individual  items,  and 
then  using  a  mathematical  formula  to  integrate  the  separate  probability  judgments. 


Impressions  tend  to  persist  even  after  the  evidence  that  created  those  impressions 
has  been  fully  discredited.  When  evidence  is  received,  there  Is  a  tendency  to  pos¬ 
tulate  a  set  of  causal  connections  that  explains  the  evidence.  Even  though  the  evi¬ 
dence  may  subsequently  be  discredited,  the  causal  links  remain  plausible  even  in  the 
absence  of  the  now  discredited  evidence. 


Consider  the  example  of  an  analyst  receiving  information  from  a  clandestine  source. 
The  analyst  may  have  formed  a  number  of  favorable  impressions  on  the  basis  of  ear¬ 
lier  reports  from  this  source.  When  the  analyst  finds  out  that  the  source  is  under 
hostile  control  and  that  the  received  information  is  probably  unreliable,  the  analyst 
will  tend  to  rationalize  earlier  impressions  by  arguing  that  the  information  is  true 
despite  the  source  being  under  control,  or  by  doubting  the  validity  of  the  report 
claiming  the  source  to  be  under  control.  In  the  latter  case,  the  phenomenon  of 
"impression  perseverance"  may  itself  affect  evaluation  of  the  evidence  that  sup¬ 
posedly  discredits  the  impression;  this  is  due  to  a  tendency  to  retain  initial  impres¬ 
sions  concerning  the  validity  of  information  am  disbelieve  new  evidence  that  con¬ 
tradicts  the  initial  impressions. 
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2.4.9  ANCHORING 


Anchoring  is  one  strategy  that  people  seem  to  use  intuitively  and  unconsciously  to 
simplify  the  task  of  mentally  processing  complex  information.  Some  natural  starting 
point  is  used  as  a  first  approximation  to  the  desired  judgment. 

This  issue  is  particularly  relevant  when  moving  into  a  new  work  setting  and  taking 
over  responsibilities  from  a  predecessor.  The  predecessor’s  analytic  estimates 
become  a  starting  point.  This  starting  point  Is  then  adjusted,  based  on  the  results  of 
additional  Information  or  analysis.  Typically,  however,  the  starting  point  serves  as  an 
anchor  or  drag  that  reduces  the  amount  of  adjustment  made,  so  that  the  final  esti¬ 
mate  remains  closer  to  the  starting  point  than  it  ought  to  be. 

Anchoring  is  a  particularly  difficult  bias  to  avoid.  Analysts  may  attempt  to  ignore 
their  previous  work  or  others’  earlier  judgments  and  re-think  the  problem  through. 
Time  and  information  constraints  may  preclude  using  this  solution.  An  alternative 
solution  might  be  the  use  of  formal  statistical  procedures.  Bayesian  statistical 
analysis,  for  example,  can  be  used  to  revise  prior  Judgments  on  the  basis  of  new 
information  in  a  way  that  is  designed  to  avoid  the  anchoring  bias. 


2.5  Memory  Aids 


2.5. 7  AUTOMATED  AIDS  AND  MEMORY  LOAD 

The  information  resources  and  variables  in  analytic  work  settings  are  usually  quite 
complex.  The  loads  imposed  on  Internal  memory  are  lessened  by  automated  and 
non-automated  memory  aidt  such  as  computerized  maps,  data  bases  and  other  refer¬ 
ence  materials.  Such  external  aids  have  advantages.  The  externalized  information 
models  they  contain  (templates,  doctrine,  IPB,  etc.)  do  not  suffer  from  memory  modifi¬ 
cation  and  judgmental  distortion  factors  that  affect  models  stored  in  the  analyst’s 
cognitive  memory.  Unfortunately,  such  materials  are  costly  to  produce,  slow  to 
update  compared  with  the  analyst’s  internal  storage  memory,  and  usually  provide  only 
a  partial  match  with  the  realities  toward  which  they  are  aimed. 

Apart  from  the  potential  analytic  value  of  automated  memory-aid  materials,  their  han¬ 
dling  and  use  can  pose  some  problems  for  the  analyst.  Passive  versions  of  such 
supports,  which  must  be  remembered  and  activated  to  be  of  use,  can  sometimes  con¬ 
tribute  to  an  analyst’s  memory  load  in  locating  materials.  Active  versions  of  such 
supports  (such  as  alarms,  forced  displays,  flashing  prompts,  rigid  reminder  schedules, 
etc.)  cars  create  interruptions,  distractions,  and  procedural  overload,  by  diverting  the 
analyst’s  limited  capacity  for  attention.  This  is  more  likely  when  the  memory  aid  is 
used  for  higher-order  processing  of  extensive  stored  intermediate  results  (problem¬ 
solving  operations).  On  the  other  hand,  active  supports  might  prove  valuable  to 
analysts  during  times  of  stress. 


2.5.2  TEAM  MEMORY 

An  external  memory  resource  widely  used  by  analysts  --  especially  under  trying  cir¬ 
cumstances  ~  Is  the  "team  memory"  represented  by  colleagues.  Team  memory  Is 
Invaluable  as  an  external  memory  supports  for  several  reasons: 
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•  Query  formulation  Is  comparatively  easy;  a  colleague  can  not  only  provide  infor¬ 
mation,  but  can  help  the  analyst  define  a  need  and  frame  a  query  in  terms  under¬ 
standable  to  the  colleague. 

•  Rapid  response  Is  available;  a  colleague  can  quickly  indicate  whether  or  not  any 
help  can  be  expected.  This  allows  the  analyst  to  search  widely  in  a  short  time  if 
necessary. 

•  Rapid  update  of  colleagues’  memory  contents  can  be  achieved  under  some  con¬ 
ditions  for  which  materials-based  memory  support  systems  would  require  consid¬ 
erably  more  time. 

•  Self  correction  of  memory  resources  is  somewhat  automatic,  since  colleagues 
tend  to  recognize  their  memory  shortcomings  and  try  to  correct  them.  While 
materials-based  memory  support  systems  could,  In  principle,  be  designed  this 
way,  it  is  not  likely  to  be  realized  in  the  near  future. 

•  Active  problem  solving  by  colleagues  Is  frequently  included  as  part  of  the  team- 
memory  services  to  one  another;  relevant  memory  contents  are  not  only  located 
and  communicated,  but  also  compared,  placed  In  contex  ts,  and  evaluated. 

The  one  disadvantage  that  may  result  from  the  use  of  team  memory  is  that  colleague 
tasks  may  become  disrupted.  This  is  often  an  inescapable  result  of  using  team- 
distributed  memory.  The  availability  of  team-distributed  memory  cannot  be 
guaranteed  under  conditions  of  high  organizational  work  load,  unless  extra  personnel 
have  been  planned  for  such  functions. 
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3.  CONCEPTUAL  MODELS 


While  the  functional  bases  of  memory  and  Information  processing  are  described  In 
terms  of  the  structures  and  processes  described  In  the  previous  chapter  (l.e.,  active 
memory,  episodic,  semantic/factual),  the  unit  of  memory  contents  is  represented  by 
the  conceptual  model. 

Conceptual  models  are  also  functional  units  In  the  sense  that  they  represent  the 
active,  currently  available  contents  of  memory  that  determine  what  Information  is  per¬ 
ceived  and  how  it  is  processed.  The  "what"  and  the  "how"  of  information  processing 
varies  depending  on  the  contents,  the  complexity,  and  the  recency  of  the  conceptual 
model  that  Is  in  active  memory  at  the  time  information  is  perceived.  The  values  of 
the  conceptual  models’  parameters  (e.g.,  complexity  and  size)  determine  the  ease  or 
difficulty  of  learning  and  remembering  new  information.  As  a  result,  they  have  an 
Important  impact  on  the  design  of  training  materials.  For  example,  training  materials 
for  new  analysts  must  oe  designed  quite  differently  than  training  materials  used  for 
the  maintenance  of  the  established  skills  of  experienced  analysts. 

The  characteristics  of  the  conceptual  model  also  impact  on  the  quality  of  communica¬ 
tion  between  people.  Since  communication  is  a  pervasive  problem,  the  implications 
derived  from  specifying  the  nature  of  conceptual  models  and  shared  conceptual 
models  can  make  a  significant  difference  to  the  analytic  community. 


3.1  The  Nature  of  Conceptual  Models 

Learning  always  occurs  within  a  given  context,  that  Is,  within  the  context  incorporat¬ 
ing  the  learning  material.  This  context  is  called  a  conceptual  model.  The  important 
point  about  conceptual  models  with  respect  to  learning  new  information  and  with 
respect  to  communicating  with  others,  is  that  conceptual  models  have  parameters 
that  have  different  values  as  a  function  of  the  amount  of  knowledge  the  person  has 
In  the  target  area. 

Conceptual  models  (CMs)  are  coherent  systems  of  knowledge  that  are  used  to 
assign  meaning  to  the  environment,  to  think,  to  remember,  and  to  solve  problems. 
CMs  serve  as  blueprints  for  aggregating  large  amounts  of  intelligence  information  into 
meaningful  higher-order  units.  The  use  of  CMs  as  information  chunking  devices  pro¬ 
vides  one  method  for  circumventing  the  limited  capacity  of  active  memory  (see  Sec¬ 
tion  2.1 ). 

Each  element  of  stored  knowledge  is  associated  with  one  or  more  conceptual  models; 
l.e.,  a  person’s  memory  does  not  contain  stray  bits  of  data  that  have  no  connection 
to  any  type  of  context,  it  Is  the  context,  3 . e . ,  the  conceptual  model,  that  determines 
how  easily  new  information  will  be  learned,  how  quickly  it  will  be  forgotten,  or  how 
new  Information  will  alter  existing  knowledge.  For  example,  if  a  person  knows  a 
great  deal  about  a  particular  topic,  new  information  associated  with  that  topic  will  be 
learned  much  more  easily  than  if  the  topic  is  unfamiliar.  Implicit  in  that  statement  is 
that  training  materials  must  be  presented  differently  In  the  two  cases. 

Conceptual  models  are  generally  organized  by  subject  matter.  Typically,  humans 
have  CMs  concerning  all  topics  that  they  have  learned  about  over  their  lifetime. 
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Obviously,  the  amount  of  knowledge  about  each  topic  varies  and  has  different 
degrees  of  relevance  to  the  person  In  question.  Tho  way  CMs  are  characterized  has 
Important  Implications  for  understanding  and  predicting  how  people  learn,  solve  prob¬ 
lems,  and  communicate  with  others  and  hence,  has  implications  for  the  design  and 
development,  of  training  materials  for  intelligence  analysts. 

The  characterization  of  conceptual  models  by  parameter  values  also  makes  it  possi¬ 
ble  to  hypothesize  how  the  memory  mechanisms  described  in  the  previous  chapter 
(e.g.,  filtering,  assimilation,  or  Interference)  Influence  information  processing 
(Welchel,  1972).  It  is  likely  that  various  cognitive  biases  are  differentially  detrimen¬ 
tal  as  a  function  of  the  parameter  values  of  the  conceptual  model  in  use  at  the  time. 
Intelligence  analysts  must  constantly  be  aware  of  the  various  cognitive  biases  that 
can  distort  their  analytic  findings.  Understanding  the  bases  for  cognitive  biases  can 
mitigate  some  of  the  circumstances  under  which  analytic  errors  can  occur.  Under¬ 
standing  the  differential  effects  of  cognitive  biases  as  a  function  of  specific  con¬ 
ceptual  models  can  alleviate  some  of  the  need  for  this  constant  vigilance. 

There  are  different  types  of  conceptual  models.  For  example,  conceptual  models  for 
language  or  for  storing  temporary  information  are  quite  different  tnan  conceptual 
models  that  deal  with  factual  knowledge.  The  present  document  deals  only  with  con¬ 
ceptual  models  that  are  made  up  o  factual  and  conceptual  information  as  used  by 
intelligence  analysts. 

Intelligence  analysis  involves  the  assignment  of  meaning  to  new  inputs  combined  with 
previously  stored  information.  Thi3  process  is  largely  concept-driven  (Norman  & 
Bobrcw,  1975)  in  that  the  analysts’  goals,  hypotheses,  and  knowledge  of  the  world 
dictate  Information  collection  priorities  and  processing/interpretation  strategies  and 
biases.  Each  analyst  has  his  or  her  own  stored  conceptions  of  the  world  that  guide 
concept-driven  processing. 

Both  semantic/factual  and  active  memory  are  organized  by  conceptual  models.  CMs 
are  stored  In  semantic/factual  memory,  but  must  be  transferred  to  active  memory 
before  they  can  be  used  to  interpret  external  or  interna!  stimuli.  Information  pro¬ 
cessing  (including  learning,  forgetting,  and  reorganization  of  xisting  information)  pro¬ 
duces  a  change  in  the  active  CM.  Information  in  active  me,  *y  is  considered  to  be 
dynamic  and  to  become  reorganized  with  new  inputs  (Hopf-Weichel,  1977),  whereas 
information  stored  In  long-term  memory  (semantic/factual  or  episodic)  is  considered 
to  be  latent.  This  view  has  implications  for  intelligence  analysis  in  that  it  impacts  on 
the  functioning  of  attention  and  vigilance.  Far  example,  environmental  stimuli  that 
are  not  related  to  an  active  CM  will  tend  to  be  ignored.  Many  cognitive  biases  can 
be  Interpreted  and  understood  within  this  framework. 

Because  of  the  role  of  CMs  as  information  chunking  devices,  an  understanding  of 
their  contents  and  characteristics  can  greatly  facilitate  the  development  of  effec¬ 
tive  training  materials. 


3.2  Characteristics  of  Conceptual  Models 

Conceptual  models  can  be  visualized  as  sets  (in  the  set-theoretic  sense)  that  over¬ 
lap  In  varying  degrees.  The  elements  of  these  sets  include: 
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•  Items  of  Information  (e.g.,  background  knowledge  concerning  the  geography  of  a 
particular  country). 

•  Relationships  among  items  (e.g.,  effect  of  weather  on  mission  effectiveness). 

•  Goals  (e.g.,  national  security,  winning  the  first  battle). 

a  Plans  for  behavior  (e.g.,  what  information  to  collect  to  achieve  goals). 

a  Sequences  of  behavior  (e.g.,  the  best  way  to  collect  Information). 

a  Time  relationships  (e.g.,  when  a  message  was  received,  or  when  a  particular 
Informational  Item  must  be  available  for  processing  within  a  given  timeframe). 

a  Access  to  other  CMs. 

a  Knowledge  concerning  the  contents  of  other  people’s  CMs.  This  is  important  in 
thut  it  allows  one  to  optimize  the  benefits  of  shared  conceptual  models  (see 
Section  3.4). 

Conceptual  models  can  be  characterized  in  terms  of  a  number  of  different  parame¬ 
ters,  with  the  value  of  each  parameter  varying  depending  on  the  CM  involved.  Some 
of  these  parameters  Include: 

a  SIZE.  This  is  an  indication  of  the  amount  of  knowledge  associated  with  the  par¬ 
ticular  CM.  The  higher  the  value,  the  more  is  known  about  the  subject  matter  of 
the  CM.  In  general,  the  larger  the  CM,  the  easier  it  is  to  add  new  items. 
Increases  In  size  come  about  when  new  information  is  added  to  an  active  CM 
(i.a.,  a  CM  that  is  in  active  memory)  or  when  information  from  two  or  more  CMs 
are  combined. 

•  COMPLEXITY.  This  Is  an  Indication  of  the  degree  of  structural  complexity  among 
the  items  within  a  CM.  This,  in  turn,  indicates  the  amount  of  understanding  asso¬ 
ciated  with  the  topic  of  the  CM.  "Understanding"  subsumes  a  knowledge  of 
Interrelationships  among  items,  their  influences,  and  potential  consequences  for 
future  events.  In  general,  a  higher  value  on  "complexity"  is  associated  with 
greater  probiem-soiving  and  decision-making  abilities  with  respect  to  the  CM. 

•  NEWNESS.  This  refers  to  how  recently  a  CM  has  become  established.  For 
example,  an  analyst  who  has  recently  been  transferred  to  a  new  country  area 
has  a  newer  CM  for  that  area  than  an  analyst  who  has  been  in  the  area  for 
several  years.  In  contrast  to  weli-estabiished  CMs,  the  newer  the  CM,  the 
greater  the  probability  that  items  associated  with  a  new  CM  will  be  forgotten.  In 
general,  new  CMs  will  have  slower  learning  rates  than  well-established  CMs. 

•  FREQUENCY.  This  refers  to  how  frequently  a  CM  is  entered  into  active  memory 
and  is  the  basis  for  practice  effects.  A  new  CM  used  frequently  rapidly 
becomes  a  well-established  CM.  In  general,  infrequently  used  CMs  (e.g.,  as 
whan  an  analyst  only  passes  through  a  country  two  or  three  times,  rather  than 
being  assigned  to  it),  will  be  associated  with  more  forgetting  and  slower  learning 
rates. 

•  "AFFECT".  Some  topics  are  of  more  interest  and/or  are  more  important  to  a  per¬ 
son  than  others.  Such  topics  have  a  higher  "affective  value",  which  is  associ¬ 
ated  with  higher  motivation,  attention,  easier  learning,  and  better  retention.  CMs 
with  higher  affective  values  also  tend  to  have  better  defined  goals  than  CMs 
with  lower  affective  values. 
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•  PLAS..CITY.  This  is  related  to  the  dynamic  nature  of  information  stored  in  active 
memory.  Plasticity  is  a  parameter  that  tends  to  be  invariant  over  all  CMs  of  a 
given  individual,  but  tends  to  vary  across  individuals.  It  refers  to  the  fact  that 
some  individuals  are  more  easily  able  to  reorganize  information  within  and 
between  CMs  than  other  individuals.  A  high  value  on  plasticity  Is  associated  with 
creativity,  as  when  a  new  solution  is  found  to  a  problem  because  the  new  solu¬ 
tion  is  simih  to  one  that  applies  >n  a  completely  different  context. 

The  parameter  values  for  any  given  CM  tend  to  be  highly  correlated  with  each  other 
and  tend  to  determine  how  well  or  easily  information  associated  with  it  is  learned, 
processed,  and  retrieved.  Ttiis  has  an  impact  on  the  development  of  training  materi¬ 
als,  on  the  issue  of  automation,  and  on  the  development  of  shared  conceptual  models 
which  form  the  basis  for  effective  communication. 

New  conceptual  models  can  be  formed  in  a  variety  of  ways: 

•  By  reorganizing  existing  CMs. 

i:  By  being  faced  with  an  important  goal  or  problem  for  which  no  adequate 
know  ige  (background,  specific,  or  procedural)  is  available. 

•  By  perceiving  stimuli  (events,  behaviors,  physical  shapes,  etc.)  in  ones  environ¬ 
ment  that  have  no  meaning  within  one’s  own  conceptual  framework.  In  general, 
however,  unfamiliar  or  strange  stimuli  are  simply  assimilated  within  existing  CMs 
(see  Section  2.1),  hence  the  differences  among  peoples’  perceptions  (and  later 
recall)  of  the  same  situation  or  event.  These  differences  in  perception  lead  to 
misunderstandings  and  poor  communication. 


3.3  Knowled-  Base  Taxonomy  for  Intelligence  Analysis 

Conceptual  Is  are  made  up  of  various  types  of  knowledge  that  are  stored  in 
semantic/fact,  i  memory.  Information  from  the  environment  is  interpreted  and 
altered  in  acti>  memory  when  it  activates  or  otherwise  interacts  with  n  CM.  This  is 
true  no  matter  how  simple  the  environmental  information,  and  no  matter  how  simple 
the  transformation.  It  is  important,  therefore,  to  identify  the  information  that 
comprises  the  anal  sts’  and  the  users’  CMs. 

A  taxonomy  of  knowledge  (see  Figure  3-1)  was  developed  based  on  a  composite  of 
answers  obtaine  on-the-job  interviews  with  analysts,  conducted  during  the  IMTIA 
project.  This  taxonomy  does  not  represent  an  exhaustive  listing  of  the  information 
encompassed  by  CMs.  One  major  concern  was  to  determine  what  types  of 
knowledge  are  required  for  analysts  to  perform  effectively  and  how  the  presence  or 
absence  of  one  type  of  knowledge  or  another  might  affect  performance. 

The  results  of  the  analyst  interviews  were  categorized  into  a  taxonomy  of 
knowledge  comprising  three  types  of  information: 

•  Specific  knowledge  refers  to  the  knowledge  necessary  to  interpret  specific 
environmental  information  and  includes  the  meaning  of  input  messages,  electronic 
signals,  or  imagery,  for  example.  These  are  typically  discrete  elements  that  may 
or  may  not  form  a  patter;  but  that  do  have  a  representation  In  memory. 

•  Background  knowledge  refers  to  the  knowledge  required  to  interpret  specific 
Information  within  the  analytic  context.  Background  knowledge  includes 
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KNOWLEDGE  TAXONOMY  FOR  THE  ANALYST 
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Figure  3-1:  Knowledge  Taxonomy  for  Intelligence  Analysts. 


information  about  a  particular  territory;  for  example,  its  geographic,  ethno¬ 
graphic,  or  doctrinal  characteristics.  Information  concerning  users,  missions,  and 
goals  are  also  aspects  of  the  background  knowledge  of  an  analyst. 

•  Procedural  knowledge  Includes  the  rules  that  are  used  during  Interpretation 
(such  as  rules  for  constructing  meaning,  for  drawee  Inferences,  and  for  making 
decisions)  as  well  as  the  cognitive  tools  and  methodologies  used  by  analysts. 


3.3.7  SPECIFIC  KNOWLEDGE 

Specific  knowledge  refers  to  the  ongoing  flow  of  information  that  an  analyst  has  to 
process.  In  processing  such  information,  analysts  look  for  meaning  and  patterns  in 
the  context  of  their  background  knowledge.  The  characteristics  of  these  patterns 
contribute  a  great  deal  to  the  reliability  and  the  validity  of  the  final  intelligence  pro¬ 
duct.  These  patterns  must  be  characterized  in  terms  of  measurable  attributes  if 
performance  on  pattern  recognition  and  interpretation  Is  to  be  evaluated  and  training 
enhanced. 

Informational  patterns  have  attributes  that  have  differential  impacts  on  analysts’ 
Interpretations  and  inferences.  The  specific  set  of  attributes  may  vary  depending  on 
the  task  and  the  goal,  but  interviews  with  analysts  indicate  the  following  attributes 
as  relevant  to  understanding  an  information  pattern: 

•  Novelty  (old  versus  new  patterns)  -  This  attribute  refers  to  whether  or  not  an 
analyst  has  experienced  a  particular  pattern  before.  A  new  pattern  may  have 
little  a  priori  meaning  associated  with  it,  compared  to  an  old  pattern,  and  this  will 
Impact  on  how  the  ^formation  Is  used  by  the  analyst.  In  general,  an  old  pattern 
would  tend  to  lead  to  a  mo.  e  reliable  Intelligence  product  than  a  new  pattern.  On 
the  other  hand,  a  new  patcern  may  lead  to  more  creative  hypothesis-  generation, 
and  could  possibly  lead  to  a  more  valid  product.  Further,  an  old  pattern  may  lead 
to  overselectivity  in  iniormation  processing  (see  Section  2.2.1),  and  thus  be 
subject  to  a  confirmatory  hias. 

•  Sparsity  (rich  versus  sparse  data)  -  This  attribute  refers  to  the  number  of  ele¬ 
ments  In  a  pattern.  Analysts  dealing  with  sparse  data  must  make  more  infer¬ 
ences  concerning  the  meaning  of  those  data.  This  tends  to  lead  to  a  product 
that  has  a  great  deal  of  uncertainty  associated  with  it.  In  particular,  sparse 
data  might  lead  soma  analysts  to  exaggerate  unusual  aspects  of  the  data, 
thereby  resulting  in  the  caricature  effect  (see  Section  2.2.2) 

•  Congruity  (congruous  versus  incongruous  information)  -  This  attribute  is  based  on 
the  assumption  that  information  patterns  represent  the  basis  for  generating 
hypotheses  concerning  the  state  of  the  world.  A  congruous  pattern  contains 
elements  that  all  tend  to  point  to  a  single  hypothesis.  An  incongruous  pattern 
can  occur  in  one  of  two  ways:  either  the  elements  suggest  more  than  one  mean¬ 
ing,  which  can  give  rise  to  several  plausible  hypotheses,  or  the  elements  tend  to 
suggest  contradictory  hypotheses.  Obviously,  congruity  in  the  informational  pat¬ 
tern  leads  to  a  more  valid  and  reliable  product  than  incongruity.  A  congruous 
pattern  can  also  be  processed  faster. 

•  Risk  (high  versus  low  risk)  -  This  attribute  refers  to  whether  or  not  the  pattern 
suggests  that  the  environmental  situation  is  risky.  This  attribute  is  important 
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because  It  may  Imply  time  constraints  on  the  performance  of  the  task,  and  it  may 
Induce  cognitive  stress,  which  could  represent  a  source  of  errors.  The  risk  fac¬ 
tor  is  obviously  an  Important  attribute  in  developing  threat  models,  but  it  is  not 
clear  whether  or  not  it  has  an  effect  on  the  validity  or  the  reliability  of  the  pro¬ 
duct. 

•  Definition  (well-defined  versus  poorly  defined)  -  This  attribute  refers  to  the  con¬ 
creteness  of  the  meanings  associated  with  the  elements  within  a  pattern,  as 
well  as  whs  the  meaning  of  the  pattern  itself.  A  well-defined  pattern  is  more 
likely  to  It  I  to  a  valid  and  reliable  product  than  a  poorly  defined  pattern.  An 
analyst's  repertoire  of  well-defined  patterns  (e.g.,  templates)  is  expanded 
through  the  acquisition  of  background  knowledge  described  in  the  next  section. 

There  are  a  number  of  issues  concerning  specific  knowledge  that  should  be  under¬ 
stood.  The  first  Is  that  the  representation  of  specific  knowledge  Is  dynamic.  That  is, 
the  information  that 's  being  received  is  constantly  changing,  so  that  knowledge  from 
memory  that  Is  brought  to  bear  in  assigning  meaning  to  this  information  has  to  be  con¬ 
tinually  updated.  This  has  significance  for  the  way  this  type  of  information  is 
represented  In  merrory.  The  second  issue  is  that  the  representation  of  specific 
knowledge  is  likely  to  be  much  more  complex  for  multi-source  than  for  single-source 
analysts,  since  multi-source  analysts  deal  with  much  more  diverse  information  than 
single  sourca  analysts. 


3.3.2  BACKGROUND  KNOWLEDGE 

Background  knowledge  provides  the  framework  for  assigning  meaning  to  specific 
informational  events.  Perception  of  real  world  events  never  occurs  in  the  abstract, 
but  always  In  terms  of  what  Is  already  known,  i.e.,  what  is  stored  in  episodic  and 
semantic/factual  memory.  The  greater  the  similarity  of  different  analysts’  back¬ 
ground  knowledge,  the  higher  the  probability  that  events  will  be  interpreted  the  same 
way,  a  consideration  that  will  be  Important  in  evaluating  performance.  It  must  be 
noted,  however,  that  even  though  perception  and  interpretation  of  environmental 
Information  is  a  function  of  the  contents  of  episodic  and  semantic/factual  memory, 
these  processes  occur  only  after  a  sub-set  of  the  available  knowledge  is  loaded  into 
active  memory.  That  is,  the  analyst  must  activate  the  appropriate  knowledge  base 
(CM)  for  that  knowledge  to  be  of  value. 

The  characteristics  of  the  background  knowledge  are  particularly  important  in  deter¬ 
mining  the  complexity  and  appropriateness  of  an  analyst’s  CM,  since  it  is  assumed 
that  the  way  meaning  is  assigned  determines  the  way  information  is  processed.  This 
is  very  much  dependent  on  the  complexity  of  the  conceptual  model,  which  in  turn  is 
highly  correlated  with  the  characteristics  of  the  background  knowledge  as  stored  in 
memory.  If  the  background  knowledge  is  extensive  in  a  particular  area  of  expertise, 
it  is  easier  to  learn  new  items  of  information  associated  with  this  area,  than  if  it  is 
not  well  developed.  Therefore,  the  characteristics  of  the  background  knowledge 
also  determine  the  rate  of  learning,  as  well  as  the  way  learning  is  transferred.  Other 
characteristics  include  the  idea  that  the  rate  at  which  the  background  knowledge 
changes  (Increases  or  decreases),  compared  to  other  types  of  knowledge,  is  rela¬ 
tively  slow.  This  imposes  constraints  on  how  procedural  knowledge  is  used  and  it 
has  the  potential  of  creating  biases  such  as  tunnel  vision. 
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A  specific  example  Involves  the  problems  encountered  by  analysts  when  they  ore 

ansferred  from  ono  theater  of  operation  to  another.  The  background  knowledge 
required  to  process  Information  (specific  knowledge)  In  the  new  area  is  necessarily 
different  than  the  background  knowledge  the  analyst  transfers  from  the  original 
theater  of  operation.  However,  building  up  a  base  of  background  knowledge  takes 
time.  Until  this  background  knowledge  is  built  up,  the  analyst  may  experience  diffi¬ 
culties  in  assigning  meaning  to  the  specific  knowledge.  The  analyst  may  also  face 
problems  if  the  available  procedural  knowledge  Is  not  directly  applicable  to  the  new 
environment. 

One  fruitful  area  for  automation  is  in  the  arcc  of  background  knowledge.  As  just 
seen  by  the  example,  the  judicious  use  of  external  memory  could  do  much  to  allevi¬ 
ate  transfer  problems.  Background  knowledge  Is  one  aspect  of  pre-processed 
knowledge,  and  external  memory  can  be  considered  as  one  aspect  of  background 
knowledge,  If  It  Is  accessible.  Training  considerations  must  Include  how  to  optimize 
the  acquisition  and  accessibility  of  background  knowledge,  whether  through  internal 
or  external  memory. 

Four  types  of  background  knowledge  relevant  to  analysis  are: 

•  Military  science 

m  Science  and  technology 

•  Sociocultural  knowledge 

•  Sensor  mode  knowledge 

Background  in  each  of  these  areas  aids  the  analyr  in  performing  most  analytical 
tusks.  Commonalities  and  differences  in  knowledge  icquirements  for  different  ana¬ 
lytic  specialities  are  highlighted  below. 

a.  Military  Science 

Commonalities  between  SIGINT,  HUMINT,  IM1NT,  and  FUSION  specialties  include  an 
extensive  knowledge  of  military  science  areas  including  Order  of  Battle  and  mili¬ 
tary  geography.  Characteristically,  this  knowledge  is  acquired  in  specific  rela¬ 
tion  to  the  targets  and  geographic  area  within  the  analyst’s  assigned  area  of 
responsibility.  The  analyst  gradually  expands  this  knowledge  base  by  cross¬ 
training  or  reassignment  to  other  intelligence  production  facilities. 

b.  Science  and  Technology 

Most  specialties  of  intelligence  analysis  use  almost  all  forms  of  physical  science 
and  technology.  A  strong  technical  background  helps  the  analyst  understand 
what  Is  seen  or  heard  in  data  collected  from  technically  complex  targets  such  as 
aircraft,  missiles,  radars,  communication  nets,  factory  complexes,  and  the  like. 
The  complex  workings  of  weapons  systems,  communications,  command  and  con¬ 
trol  organizations  and  procedures,  and  electronics  systems  must  be  understood 
for  proper  interpretation  of  intelligence  data  in  both  strategic  and  tactical  mis¬ 
sions. 

Emphatically,  this  principle  holds  for  all  levels  of  intelligence  analysis  as  defined 
earlier.  Although  the  inexperienced  mage  interpreter  performing  a  target  loca¬ 
tion  or  counting  function  requires  much  less  general  knowledge  and  understand¬ 
ing  than  an  analyst  w'ib  a  more  explicit  evaluation  role,  all  levels  benefit  by  a 
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thorough  understanding  In  their  area  of  responsibility.  The  analyst  expands  this 
area  of  knowledge  through  exposure  to  new  systems  and  through  on-going 
Interests  In  technical  areas  and  activities. 

c.  Sociocultural  Knowledge 

Knowledge  of  a  target  country’s  culture  Is  valuable  In  discriminating  militarily  sig¬ 
nificant  items  from  non-signlflcant  ones.  Ethnography  helps  the  analyst  know 
where  to  look  for  significant  items  and  what  to  filter  out.  Sociocultural 
knowledge  Is  particularly  Important  in  strategic  intelligence  analysis  where  the 
use  of  economic,  political,  and  industrial  models  may  be  Involved. 

d.  Sensor  Mode  Knowledge 

Because  the  entire  intelligence  production  cycle  is  included  in  the  scope  of 
Intelligence  analysis  activities,  the  ability  to  understand  the  capabilities  of  sen¬ 
sor  systems  Is  a  fundamental  knowledge  requirement.  Although  an  Imaqery  inter¬ 
preter  may  not  need  to  know  how  a  particular  camera  works,  the  constraints 
under  which  photographic  data  Is  collected  must  be  understood  when  making 
decisions  about  collection  plans  and  the  quality  of  Intelligence  products. 


3.3.3  PROCEDURAL  KNOWLEDGE 

The  most  important  aspects  of  procedural  knowledge  are  decision  making  skills  and 
the  implicit  and  explicit  rules  and  strategies  used  to  perform  the  tasks  of  intelligence 
analysis.  Procedural  knowledge  also  includes  specific  techniques  and  methods  used 
In  analytic  planning,  estimation,  forecasting,  and  so  forth. 

Although  procedural  knowledge  has  to  be  learned,  and  experience  plays  a  major  role 
In  Its  effective  use,  It  Is  hypothesized  that  this  type  of  knowledge  is  more  easily 
transferred  from  one  theater  of  operation  to  another,  than  is  either  speciiic  or  back¬ 
ground  knowledge,  it  Is  further  hypothesized  that  there  are  fewer  differences 
between  single-  and  multi-source  analysis  in  the  way  procedural  knowledge  is  used 
than  in  the  way  other  types  of  knowledge  are  used. 

Procedural  knowledge  is  valuab'e  only  to  the  extent  that  it  enables  the  analyst  to 
manipulate  Information.  Although  skilled  individuals  typically  possess  a  large  store  of 
Information  about  their  field  of  expertise,  there  has  been  no  strong  theoretical 
analysis  showing  how  the  existence  of  a  large  store  of  information  contributes  to 
successful  problem  solving  (Greeno,  1978). 

Procedural  knowledge  provides  a  bridge  between  specific/background  knowledge 
and  the  procedures  acting  on  this  knowledge  base  that  enable  problem-solving  and 
task-performance  activities. 


3.4  Shared  Conceptual  Models  in  Communications 

Shared  conceptual  models  (SCMs)  have  two  major  components: 

•  Those  aspects  of  a  conceptual  model  that  two  or  more  people  have  in  common, 
such  as  a  language  or  the  knowledge  of  a  subject  matter. 
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•  Accoa.s  cutis  ami  knowledge  concerning  the  contents  of  another  person's 
coptual  modols.  (This  is  the  component  that  makes  team  memory  ami  externa! 
memory  effective  as  an  extension  of  one’s  own  memory.) 

Communication  Is  Impossible  without  SCMs  and  analysts  and  the  users  of  intelligence 
must  be  encouraged  to  develop  SCMs  concerning  the  analytic  situation.  The  threat 
model,  discussed  In  Chapters  6  and  7,  Is  to  be  used  as  the  main  SCM  for  intelligence 
analysis. 

In  using  SCMs,  it  Is  important  to  remember  that  these  modols  ensure  that  a  given 
input  means  the  same  to  those  Involved  in  communicating  about  the  input.  They 
reduce  the  perceptual  and  cognitive  differences  between  people.  From  this  point  of 
view,  SCMs  increase  the  effectiveness  cf  communication  by  reducing  the  amount  of 
time  needed  for  communicating.  On  the  other  hand,  there  are  advantages  in  the  per¬ 
ceptual  and  cognitive  differences  that  humans  experience  >n  assigning  meaning  to 
their  environments.  These  are  the  differences  that  are  responsible  for  richer 
problem-solving  behavior  in  brainstorming  sessions,  for  example,  or  that  lead  to  more 
diverse  hypothesis  generation.  The  achievement  of  a  proper  balance  between 
developing  SCMs  and  encouraging  perceptual  and  cognitive  differences  between 
analysts  Is  a  matter  for  further  research,  in  general,  however,  shared  conceptual 
models  should  be  encouraged  for  an  optimal  analyst-client  relationship. 

Without  prior  conceptual  preparation,  communications  during  battle  woulc  have  to  be 
fully  elaborated  in  order  to  convoy  full  meaning,  and  for  all  participants  to  have  a 
complete  understanding  of  their  tasks  and  the  goals  of  the  mission.  Under  such  con¬ 
ditions,  SCMs  would  be  minimal,  and  the  only  way  to  ensure  accuracy  and  a  complete 
understanding,  would  be  through  direct  feedback.  On  the  other  hand,  if  battlefield 
participants  take  prior  care  to  develop  SCMs,  the  amount  of  needed  data  flow  for 
battlefield  communication  can  be  curtailed. 

Many  communication  errors  are  the  result  of  interpreting  information  within  one’s  own 
personal  context  or  CM,  which  may  either  be  wrong  for  the  given  situation,  or  may 
simply  be  different  than  that  of  the  communicator.  Thus,  an  additional,  and  very 
Important  by-product  of  SCMs,  is  a  reduction  in  communication  errors  and  misunder¬ 
standings,  since  SCMs  Include  context  as  a  shared  factor. 

In  promoting  the  development  of  SCMs,  there  are  Issues  for  the  analyst  to  consider, 
as  well  as  issues  for  the  user  of  intelligence. 

Issues  for  the  analyst: 

•  What  does  the  user  know? 

•  What  does  the  user  need  to  know? 

—  What  information  should  be  left  out? 

—  Whet  information  should  be  included? 

—  What  level  of  detail  should  be  presented? 

—  Frequency  of  update/refresh  known  Information? 

•  How  does  the  user  react  to  uncertainty? 

•  How  s‘  mid  uncertainty  be  conveyed? 
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•  Has  the  transmitted  message  become  part  of  the  SCM?  (Has  it  been  clearly 
understood  by  the  user?) 

Issues  for  the  user: 

•  The  user  must  remember  to  provide  feedback  to  the  analyst  that  gives  the 
analyst  the  following  lnformation: 

—  The  user's  perception  of  threat. 

—  The  user's  reaction  to  uncertainty. 

—  The  user's  reaction  to  new  information: 

—  Is  it  adequate? 

—  Has  it  altered  the  perception  of  threat? 

—  Ha3  it  become  a  part  of  the  SCM? 

—  What  add!t:onal  Information  is  needed? 

Issues  for  both  the  ane  yst  and  the  user  include  the  need  to  develop  a  shared 
understanding  of  the  meaning  and  .lterpretation  of  uncertainty  and  of  probability 
estimates. 

There  are  many  research  and  development  issues,  as  well  as  training  implications, 
associated  with  the  concepts  of  conceptual  models  and  shared  conceptual  models. 
These  will  be  discussed  in  Chapter  9  of  this  document. 
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4.  SKILL  REQUIREMENTS  FOR  INTELLIGENCE  ANALYSIS 


In  Chapters  2  and  3,  the  theoretical  bases  for  intelligence  analysis  were  described, 
Including  the  cognitive  structures  and  processes  that  comprise  the  cognitive  model. 
During  the  IMTIA  interviews  with  intelligence  analysts  and  as  a  result  of  examining 
analysts’  answers,  work  environments,  tasks,  and  missions,  Logicon  researchers 
found  that  there  are  certain  basic  skills  that  ail  good  analysts  possess.  These  basic 
skills  comprise  general  analytic  skills  which,  when  combined,  serve  to  fulfill  the  two 
primary  goals  of  an  intelligence  analyst,  namely: 

1.  To  reduce  uncertainties  for  the  users  of  intelligence. 

2.  To  maximize  the  use  of  scarce  resources. 

These  two  goals  serve  to  increase  the  amount  of  control  that  the  users  of  intelli¬ 
gence  have  over  the  threat  situation.  To  increase  the  probability  of  fulfilling  the 
analytic  goals,  the  following  four  basic  skills  are  required: 

•  Conceptual  modeling. 

•  Information  triage. 

•  Decision  making. 

•  Effective  communications. 


4.1  Conceptual  modeling 

The  structure  of  conceptual  models  was  described  in  Chapter  3.  Conceptual  model¬ 
ing  is  also  a  general  skill  for  analysis  in  that  it  forms  the  basis  for  effective  triage, 
decision  making,  and  effective  communications.  Conceptual  models  are  the  means  by 
which  the  analyst  organizes,  stores,  and  maintains  the  extensive  knowledge  base 
that  is  required  for  inteliigence  analysis.  Conceptual  models  provide  tne  base  for: 

•  Interpretation  of  sparse  data. 

•  Fill-in  of  missing  or  inaccurate  data. 

•  Comparison  of  alternatives. 

•  Communications. 

•  Predictions. 

•  Access  to  external  memory. 

Among  the  many  different  skills  subsumed  by  conceptual  modeling,  pattern  analysis 
and  recognition  are  particularly  important  in  inteliigence  analysis. 


During  normal  perception,  the  human  cognitive  system  routinely  "filters"  out  most  of 
the  millions  of  bits  of  sensory  data  reaching  the  sense  receptors;  yet  despite  this 
filtering,  visual  perception  is  relatively  efficient.  One  reason  for  this  is  the  degree  of 
redundancy  in  the  input  data  stream,  and  the  fact  that  the  human  cognitive  system  is 
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able  to  reconstruct  from  long-term  memory  aspects  of  a  percept  that  may  not  be 
directly  available  from  sensory  data.  The  intelligence  analyst  faces  a  more  difficult 
task,  however,  because  the  amount  of  redundancy  in  the  data  is  not  known.  Yet,  as 
In  the  case  of  visual  perception,  analysis  can  be  efficient  if  the  relevant  conceptual 
model  (CM)  can  be  retrieved  from  long-term  memory.  This  can  then  be  used  to  "fill- 
in"  the  parts  of  the  "intelligence  picture"  and  direct  information  selection  along  other 
paths  and  different  sources. 

Intelligence  analysts  are  often  involved  in  routine,  rule-based  recognition  that  prob¬ 
ably  occurs  "automatically"  as  the  incoming  information  >s  found  tc  fit  the  analyst's 
CM  more  and  more  closely.  The  CM,  which  is  multi-faceted  and  frequently  contains 
numerous  related  schemata  or  templates,  imposes  meaning  on  the  incoming  informa¬ 
tion  according  to  the  analyst’s  experience,  intentions,  motivation,  and  goals,  and  is 
activated  by  the  incoming  messages.  But  what  happens  when  the  data  do  not  fit  any 
of  the  information  within  the  CM?  Or  if  the  situation  is  so  puzzling  that  no  CM  sug¬ 
gests  itself?  Which  CMs  are  activated,  or  which  schemata  within  the  CM  are 
developed  in  this  case?  The  development  of  the  appropriate  schema  structure  is 
dependent  upon  a  number  of  non-attentional  factors,  such  as  the  analyst’s; 
knowledge  base,  effectiveness  of  memory  search,  etc.  However,  attentional  factors 
are  important  in  two  ways:  (1)  in  the  "intensive"  sense  of  attention;  that  is,  effortful 
processing  and  consideration  of  message  features  and  combinations  of  features. 
Successful  analysis  demands  active,  memory-driven  application  of  attentional 
resources  to  the  development  and  testing  of  an  appropriate  CM.  (2)  The  develop¬ 
ment  of  the  appropriate  CM  is  dependent  upon  the  selection  of  the  correct  informa¬ 
tion  sources  (through  an  attention  allocation  policy,  and  appropriate  chunking,  as 
suggested  above)  while  the  selection  policy  is  itself  influenced  by  the  initial,  frag¬ 
mentary  CM  that  the  analyst  possesses. 

Consider  the  analogy  of  solving  a  jigsaw  puzzle.  If  the  picture  of  the  completed  puz¬ 
zle  is  provided  then  a  schema  already  exists  to  direct  the  search  for  particular 
pieces,  and  the  successful  completion  of  the  puzzle  merely  requires  the  matching  of 
Individual  pieces  with  parts  of  the  given  picture.  If  no  picture  is  provided,  the  puzzle 
is  more  difficult  (although  by  no  means  as  difficult  as  the  puzzles  faced  by 

analyst’s).  One  must  examine  a  piece  or  group  of  pieces  for  context  clues  so  that  an 

initial  conceptual  mode!  can  be  developed.  This  model  then  directs  the  search  for 

particular  pieces,  some  of  which  might  lead  to  the  rejection  of  the  initial  model  and 

the  development  of  another  one,  in  a  feedback  loop.  The  conditions  under  which  this 
feedback  loop  leads  to  successful  analysis  or  results  only  in  a  "vicious  circle"  are 
unknown.  Furthermore,  the  analyst  faces  another  problem  that  does  not  arise  with 
Jigsaw  puzzles.  With  jigsaw  puzzles  (as  with  ail  other  puzzles),  one  knows  that  there 
must  be  a  solution.  Suppose,  however,  that  a  perverse  toy  manufacturer  were  to 
market  a  jigsaw  puzzle  of  an  abstract  painting  and  not  show  the  "correct  solution"; 
then  the  comptated  picture  would  vary  in  as  many  ways  as  one’s  concept  of  what 
abstract  art  should  look  like.  The  analyst’s  task  lies  somewhere  between  the 
extremes  of  this  situation  and  that  of  the  standard  jigsaw  puzzle  with  a  given  pic¬ 
ture.  A  "solution"  is  possible  only  insofar  as  the  final  pattern  "makes  sense",  or  is 
thought  to  be  probable. 

Performanca  in  the  above-mentioned  situation  is  limited  by  attentional  factors.  First, 
even  though  the  analyst’s  conceptual  mode!  and  therefore  his  search  strategy  may 
be  appropriate,  the  fact  that  he  can  only  direct  his  attention  to  certain  items  at  any 
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one  time  may  mean  that  on  certain  occasions  an  item  necessary  to  confirm  a  particu¬ 
lar  hypothesis  is  missed.  Second,  research  on  the  cognitive  psychology  of  problem 
solving  has  shown  that  if  certain  critical  cues  are  not  attended  to  during  the  early 
phase  of  problem  solving,  the  likelihood  Is  that  they  will  not  be  picked  up  later  so 
that  the  probability  of  correct  solution  is  small  (Wickelgren,  1974).  Since  the  capa¬ 
city  for  sustained  attention  for  critical  cues  is  likely  to  drop  with  time,  successful 
pattern  recognition  requires  that  attention  be  directed  to  the  relevant  cues  as  soon 
as  possible  in  the  analytic  cycle. 

Perceptual  skills  are  especially  important  for  single-source  intelligence  production. 
There  is  a  strong  consensus  in  the  field  that  the  perceptual  skills  fundamental  to 
SIGINT  and  IMINT  single-source  production  must  be  developed  by  using  real  data. 
That  is,  the  refined  perceptual  skills  necessary  to  interpret  particular  kinds  of 
images  or  to  recognize  certain  voices  of  certain  morse  or  telegraphic  operators  can¬ 
not  be  fully  developed  in  exercises  dissociated  from  real  targets. 

Familiarity  with  particular  target  areas  increases  analysts’  effective  use  of  percep¬ 
tual  and  cognitive  skills,  their  confidence  and  speed  in  interpretation,  and  their  ability 
to  detect  significant  changes.  A  substantial  time  period  is  required  to  gain  such  fam¬ 
iliarity  even  for  experienced  analysts.  SIGINT  analysts  typically  state  that  it  takes 
two  to  six  months  to  adapt  to  a  new  target  area  after  being  reassigned.  IMINT 
analysts  are  often  semi-permanently  assigned  by  geographic  area  and/or  target 
types  in  order  to  take  advantage  of  the  resulting  accumulation  of  perceptual  familiar¬ 
ity. 


4.2  Information  Triage 

Information  triage  Is  the  means  whereby  the  mass  of  available  information  is  sorted 
ter  relevance.  Information  overload  is  a  formidable  problem  for  the  analyst  dealing 
with  the  target-rich  modern  battlefield  and  modern  collection  systems.  The 
unprepared  analyst  could  easily  be  inundated  by  message  flow  and  be  unable  to  per¬ 
form  effectively. 

The  relevance  of  information  cannot  be  defined  in  the  abstract.  Rather,  it  has  to  be 
defined  In  terms  of  the  goals  and  subgoals  of  the  intelligence  community,  the  various 
echelons  within  Army  intelligence,  and  the  specific  job  of  a  particular  intelligence 
analyst.  How  relevance  varies  according  to  the  goals  of  a  nation  is  described  by 
Codevilla  (1980),  in  a  paper  dealing  primarily  with  the  type  of  organization  required 
for  effective  intelligence  analysis.  However,  both  the  organization  and  the  relevance 
of  intelligence  data  are  shown  to  relate  to  the  needs  and  goals  of  different  govern¬ 
ments.  As  Codevilla  points  out: 

A  nation  engaged  in  political  or  military  offensives  has  a  different  analtfical  problem 
than  a  nation  on  the  receiving  end  of  such  offensives.  A  government  that  does  not 
believe  it  is  in  danger  of  military  defeat  is  likely  to  focus  its  analytical  efforts  dif¬ 
ferently  than  a  government  that  believes  such  a  defeat  is  possible. 

Thi3  quote  highlights  the  importance  of  goals  in  determining  the  mission  requirements 
and  the  relevance  of  the  information  needed  to  fulfill  those  requirements.  Numerous 
other  factors  influence  how  relevance  for  information  triage  is  determined.  For 
example,  the  process  of  triage  depends  on  the  avallal  'Ity  of  resources,  the  require¬ 
ments  of  specific  use  ;,  and  the  work  setting  context,  its  successful  completion  is  a 
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function  of  the  more  specific  skills  of  attention  in  information  selection. 


There  are  several  varieties  of  attention,  all  of  which  share  in  common  the  idea  of 
conscious  processing  of  information.  Here,  the  source  of  information  may  be  external 
(data-driven)  or  internal  (memory-driven)  (Kahneman,  1973;  Norman  and  Bobrow, 
1975)  The  different  components  of  attention  may  be  thought  to  differ  along  two 
dimensions,  the  intensive  and  the  directional.  Attention  can  be  directed  to  one  or 
other  source  (internal  or  external),  and  it  may  be  deployed  with  more  or  less  inten¬ 
sity.  Listed  below  are  other  dimensions  by  which  different  varieties  of  attention  may 
be  Identified: 

•  Source:  Focussed  or  Divided  Attention. 

•  Processing  Resources:  Required  (Controlled)  or  Not  (Automatic). 

•  Duration:  Transient  or  Sustained  Attention. 

Analysts  develop  appropriate  attention  allocation  policies  that  determine  how  they 
select  information  from  the  stream  of  input  data  they  process.  One  such  policy  may 
be  to  attend  only  to  high  information-value  or  high-probability  sources.  Under  normal 
conditions,  this  implies  that  low-value  sources  may  be  "filtered  out"  without  serious 
consequence.  Under  stress  conditions,  these  tow-value  sources  are  ignored;  thus  in 
non-routine  conditions,  where  "low-value"  sources  carry  important  information,  errors 
may  result. 

From  interview  data  gathered  from  intelligence  analysts,  we  have  concluded  that 
Intelligence  analysis  is  predominantly  concept-driven  rather  than  data-driven.  This 
conclusion  was  reached  after  observing  the  extensive  memory  resources  that 
analysts  orlng  to  the  production  of  most  intelligence  products,  and  noting  that  much 
of  the  memory-based  information  was  conceptual  in  nature.  The  production  process 
is  also  concept-driven  in  the  sense  that  intelligence  products  can  be,  and  often  are, 
produced  in  the  absence  of  any  data.  Intelligence  preparation  of  the  battlefield 
(IPB)  Is  an  example  of  an  activity  that  depends  almost  entirely  on  previously 
developed  and  stored  knowledge.  As  indicated  earlier  in  this  report,  highly  experi¬ 
enced  and  effective  analysts  often  appear  to  organize  the  mental  storage  of  such 
knowledge  around  conceptual  models.  The  analyst’s  ability  to  deal  with  conceptual 
models  grows  as  a  result  of  experience  in  learning  abstract  concepts.  The  superior 
analyst  is  one  who  carries  out  data  aggregation  in  a  manner  likely  to  lead  to  the 
detection  of  a  sign:  leant  pattern.  He  does  not  have  access  to  more  or  better  infor¬ 
mation  than  the  less  able  analyst,  but  is  better  able  to  chunk  the  available  items  of 
information  (including  "items"  drawn  from  memory)  into  significant  wholes.  This 
Implies  that  "bigger  and  better"  information  collection  systems  do  not  necessarily 
result  in  better  analysis. 


4.3  Decision  Making 

Decision  making  is  a  general  skill  that  underlies  almost  all  information  processing. 
While  decision  making  can  be  performed  by  following  prescribed  normative  pro¬ 
cedures,  any  analytical  decision  making  activity  must  be  managed  and  regulated  by 
the  goal  hierarchy  of  the  intelligence  community. 
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While  analysts  do  not  make  command  decisions,  critical  decisions  are  made  at  nil  lev¬ 
els  of  analysis,  especially  decisions  to  include  or  exclude  data  from  analysis. 
Analysts  are  constantly  called  on  to  make  decisions  regarding  what,  how,  and  when 
Information  can  be  used  to  support  strategic  or  tactical  decisions  made  by  the  users 
of  this  information.  Despite  this  critical  role,  analysts  are  generally  not  trained  as 
decision  makers. 

Analytic  decision-making  involves  a  set  of  unique,  complex,  and  demanding 
Information-processing  and  problem-solving  tasks.  Typically,  the  decision  maker  must 
gather  and  evaluate  various  sources  of  evidence  and  enumerate  possible 
hypotheses,  analyze  the  consequences  of  particular  decisions,  and  make  recommen¬ 
dations  where  necessary  on  possible  courses  of  action.  Almost  always  tho  informa¬ 
tion  needed  to  make  reliable  decisions  Is  insufficient,  even  when  augmented  by 
memory  or  decision  aids,  and  the  possible  options  open  to  the  decision-maker  are  not 
clearly  specified.  The  critical  nature  of  this  difficult  task  is  heightened  when  the 
time  available  for  making  decisions  is  limited,  when  the  task  has  to  be  performed 
under  other  stress-inducing  conditions,  and  when  the  courses  of  action  are  associ¬ 
ated  with  high  risks. 


4.3. 1  CONCEPTUAL  STRUCTURE  OF  DECISION-MAKING 

Any  decision-making  task,  no  matter  how  simple,  can  be  divided  up  into  two  basic 
stages: 

•  Problem  Structuring 

•  Alternative  Selection 

In  the  first  stage  the  possible  hypotheses  under  consideration  bv  the  decision-maker 
are  identified  and  assessed,  or,  if  initially  unavailable,  created  (hypothesis  genera¬ 
tion).  In  the  second  stage  choices  between  hypotheses  or  possible  courses  of 
action  are  made. 

Although  this  distinction  has  been  recognized  for  some  time,  the  problem  structuring 
and  hypothesis  generation  phases  of  decision-making  have  been  comparatively 
neglected  in  the  research  literature.  Both  of  these  are  important  aspects  of 
decision-making  in  intelligence  analysis.  Katter,  Montgomery,  and  Thompson  (1979), 
who  conducted  several  in-depth  interviews  with  intelligence  analysts  involved  in 
both  single-source  and  multi-source  intelligence  activities,  found  that  analysts  view 
themselves  as  "detectives"  or  "historians"  faced  with  the  problem  of  solving  com¬ 
plex  "puzzles"  and  "mysteries".  Before  detectives  or  historians  can  solve  a  particu¬ 
lar  puzzle,  they  must  think  about  the  problem,  formulate  some  analytical  structure, 
and  develop  possible  hypotheses  or  solutions.  Only  then  can  they  profitably  evalu¬ 
ate  the  evidence  and  begin  to  arrive  at  the  most  satisfactory  solution.  Although 
Intelligence  problems  rarely  have  "perfect  solutions",  successful  intelligence 
analysis  also  begins  with  the  recognition  and  structuring  of  a  decision  problem. 


4.3.2  PROBLEM  STRUCTURING  AND  HYPOTHESIS  GENERATION 

Problem  structuring  consists  of  translating  the  decision  problem  into  a  formal  struc¬ 
ture  by  relating  the  aspects  of  the  problem  to  the  elements  of  a  formal  model. 
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Keeney  and  Raiffa  (1976)  identify  probiem  structuring  with  the  complete  specifica¬ 
tion  of  decision  choices  and  objectives.  Edwards’  (1977)  so-called  SMART  model, 
for  example,  consists  of  a  set  of  choices  available  to  the  decision-maker,  a  set  of 
objectives,  and  a  set  of  attributes  with  which  the  choices  are  evaluated;  in  addition 
there  are  several  criteria  that  these  sets  need  to  satisfy.  SMART  is  an  additive, 
riskless  model  <>f  multiattribute  preferences;  it  was  found  to  be  useful  for  modeling 
certain  aspect  of  public  utility  measurement  in  social  decision-making.  Like  other 
models,  however,  its  analytical  accuracy  depends  on  a  number  of  assumptions.  The 
major  difficulty  arises  in  ensuring  that  ail  aspects  of  the  decision  problem  and  all  ele¬ 
ments  of  the  model  have  been  adequately  specified. 

All  problems  can  be  structured  and  analyzed  using  a  general  model  comprising  four 
components: 

•  All  problems  begin  with  an  initial  state.  This  is  the  condition  of  things  when  the 
problem  is  recognized  and  the  task  is  to  recognize  all  the  relevant  factors  per¬ 
taining  to  the  initial  problem  state.: 

•  All  problems  have  a  goal  state.  This  is  the  condition  of  things  when  the  problem 
Is  solved.  It  is  the  desired  end  state  and  the  task  is  to  identify  the  differences 
between  the  initial  and  the  end  state  on  the  relevant  factors. 

•  Progress  is  made  from  the  initial  state  to  the  goal  state  by  way  of  the  solution 
path.  The  solution  path  is  the  sequence  of  steps  to  be  followed  to  achieve  a 
solution.  There  may  be  more  than  one  solution  path  and  the  task  is  to  identify 
which  may  be  the  best  for  the  problem,  especially  when  resources  (e.g.,  time) 
are  scarce. 

•  The  elements  of  the  problem  are  embedded  in  a  context.  The  context  may 
include  parallel  problems,  resources  available,  country  areas,  and  overall,  guiding 
goals  of  the  analyst  or  the  user. 

This  general  problem-solving  model  is  simply  a  structuring  tool  used  in  conjunction 
with  the  structuring  and  generation  of  hypotheses. 

Hypothesis  generation  is  a  process  that  ideally  requires  the  decision-maker  to  gen¬ 
erate  a  set  of  all  passible  hypotheses  that  pertain  to  the  problem.  In  general,  how¬ 
ever,  most  persons  have  a  great  deal  of  difficulty  in  doing  so.  Gettys  and  his  col¬ 
leagues  have  reported  a  number  of  studies  showing  that  the  hypothesis  set 
developed  is  most  often  deficient  (not  exhaustive),  and  that  too  few  hypotheses  are 
retrieved  from  memory  (Gettys  and  Fisher,  1979;  Mehle,  1980).  Mehle  (1980)  found 
that  both  experienced  mechanics  and  novices  had  difficulty  in  generating  complete 
hypothesis  sets  in  response  to  a  decision  task  concerning  a  non-functioning  automo¬ 
bile.  Doth  groups  of  subjects  were  also  overconfident  in  their  subjective  estimates 
of  the  probability  of  any  hypothesis  in  their  hypothesis  set. 

These  phenomena  apply  equally  to  naive  observers  as  they  do  to  intelligence 
analysts  or  other  expert  decision-makers.  An  analyst  will  arrive  at  a  hypothesis  with 
little  or  no  data.  Subsequent  information  gathering  tends  to  increase  the  analyst’s 
confidence  in  the  original  hypothesis  rather  than  the  accuracy  of  the  hypothesis 
(Heuer,  1  978). 

The  ail  source  analyst  has  particular  difficulties  In  hypothesis  generation  and  evalua¬ 
tion,  compounding  the  overall  problems  underlying  decision-making.  Some  of  these 
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additional  problems  include: 

•  Hypotheses  have  already  been  generated  by  the  information  sources  (normally 
by  other  analysts). 

•  Hypotheses  were  generated  when  the  collection  plans  we:e  formulated. 

•  Hypotheses  were  made  when  previous  data  were  collected. 

All  of  these  difficulties  may  limit  the  hypothesis  generation  capabilities  of  the  analyst 
and,  in  order  to  overcome  these  limitations,  the  analyst  may  be  forced  to  revert  to 
the  original  source  data  and  formulate  a  new  hypothesis. 

Although  something  is  known  of  these  and  other  factors  that  limit  hypothesis  genera¬ 
tion,  the  underlying  processes  themselves  are  not  fully  understood.  Gettys  and 
Fisher  (1979)  propose  that  hypothesis  generation  involves  retrieval  from  memory 
from  possible  states  of  the  world,  the  retrieval  cues  being  elicited  by  the  initial  infor¬ 
mation  provided.  They  propose  that  these  hypotheses  are  rapidly  checked  for  con¬ 
sistency  against  the  remaining  information.  The  existence  of  a  high-speed  verifica¬ 
tion  process  i.i  hypothesis  generation  was  suggested  by  the  results  of  a  number  of 
studies  carried  out  by  Gettys  and  his  colleagues. 

The  identification  of  two  independent  cognitive  processes  -  retrieval  from  memory 
and  consistency  checking  -  to  underly  hypothesis  generation  is  an  important  contri¬ 
bution.  This  two-process  view  may  be  especially  applicable  to  the  further  under¬ 
standing  of  how  hypotheses  change  as  new  information  is  received  in  decision  situa¬ 
tions  with  multiple  sources  of  information.  However,  a  number  of  other  questions  con¬ 
cerning  hypothesis  generation  remain.  An  important  cognitive  bias  that  influences 
hypothesis  generation  is  the  confirmation  bias  (see  Section  2.4.4.)  Persons  are 
reluctant  to  seek  disconfirmlng  evidence  against  a  hypothesis  they  hold,  particularly 
If  their  belief  in  the  hypothesis  Is  quite  strong.  Quite  often  such  a  hypothesis  might 
be  the  initial  h-  "  esis  that  the  decision-maker  thought  of.  Since  the  initial 
hypothesis  ma>  h*»  very  resistant  to  disconfirmation,  it  is  Important  to  understand 
both  how  initi"'  *ses  r  formed  and  why  people  tend  to  hold  to  them  strongly. 


4.3.3  ALTERNAhJE  SELECTION 

Alternative  selection  is  one  component  of  the  decision-making  task  that  has  been 
extensively  studied  and  for  which  normative  models  have  been  developed  (e.g., 
Raiffa,  1968;  Slovic,  Fischhoff,  &  Lichtenstein,  1977).  Here,  a  specific  choice  of 
alternatives  (resulting  from  the  hypothesis  generation  phase)  is  presented  to  some¬ 
one  who  must  then  select  one  course  of  action.  It  is  a  very  difficult  psychological 
task  to  compare  several  courses  of  action  and  to  select  one,  especially  in  the  com¬ 
plex  environment  of  intellic  a..  ‘s.  First,  it  strains  the  limited  capacity  of 

short-term  memory  sk •.  -  vise  a  single  alternative  and  its  implications,  let 
alone  carry  several  in  mind  simultaneously  in  order  to  compare  them.  Second,  if  the 
alternatives  are  complex,  there  are  no  clear  methods  of  comparison,  even  if  the 
several  choices  can  be  viewed  simultaneously.  And  third,  there  are  always  unknown 
factors  that  intrude  upon  the  sltuat'  Some  of  the  results  of  the  action  are 

hypothetical  In  that  there  is  frequenti'  way  of  knowing  what  will  happen  if  such 

an  action  is  selected.  Some  of  th  osults  of  the  decision  may  depend  on  the 
enemy’s  reaction  to  it,  or  on  some  other  unknown  factors. 
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The  function  of  rational  decision  theory  is  to  identity  the  information  that  is  relevant 
to  a  decision  and  to  specify  how  the  information  should  be  put  together  to  arrive  at  a 
conclusion.  The  major  principle  of  rational  decision  making  is  that  of  optimization :  All 
other  things  being  equal,  pick  the  alternative  with  the  greatest  value,  by  no  means 
an  easy  task  considering  the  numerous  uncertainties  that  analysts  constantly  deal 
with. 

The  distinction  between  problem  structuring  and  alternative  selection  provides  the 
key  to  understanding  the  role  of  aids  for  enhancing  analytic  capabilities.  Biases 
affecting  the  generation  of  hypotheses  are  different  from  those  which  affect  t.ie 
evaluation  of  decision  alternatives,  although  some  biases  are  present  in  both  stages. 
The  cognitive  biases  influencing  alternative  selaction  for  a  given  set  of  hypotheses 
have  been  well  documented.  The  highest  pay-off  would  come  in  the  exploitation  of 
aids  for  alternative  selection.  Cognitive  biases  influencing  the  generation  of 
hypotheses  and  the  manner  in  which  hypotheses  are  maintained  and  discarded  are 
only  beginning  to  be  explored.  Thus,  different  considerations  apply  in  developing 
techniques  for  counteracting  biases.  Different  decision-aid  methodologies  may  be 
appropriate  depending  on  the  component  of  the  decision  problem  involved. 


4.4  Effective  Communication 

Without  effective  communication  no  intelligence  activity  or  process  can  be  optimally 
utilized.  The  analyst  must  communicate  in  many  different  ways,  being  dependent  on 
communications  skills  for  tasking,  utilization  of  external  knowledge  sources,  recordirg 
the  analytic  procedure,  and  product  delivery.  Many  of  the  problems  that  currently 
exist  in  the  intelligence  community  are  related  to  communication  problems.  Effective 
communications  must  involve  an  understanding  of  the  communications  networks  and 
the  media  available.  The  analyst  must  also  know  how  to  control  the  data  flow  for 
effective  transfer  of  information. 

Effective  communication  can  be  optimized  by  a  process  that  involves  the  develop¬ 
ment  of  shared  conceptual  models.  Conceptual  Models  (CMs)  and  Shared  Conceptual 
Models  (SCMs)  are  central  to  an  understanding  of  learning,  information  processing, 
remembering,  and  communicating,  as  discussed  In  Chapter  3.  These  models  are 
o  lanizatlonal  systems  In  memory  that  attribute  meaning  to  informational  events  and 
that  are  responsible  for  making  communication  possible.  The  analyst  must  be  aware 
of  and  underst  nd  differences  in  the  perspectives  and  analytical  models  employed 
by  fellow  analysts  and  the  users.  The  relatively  static  features  of  a  shared  concep¬ 
tual  model  form  an  implicit  reference  base  for  communications;  and  relatively  dynamic 
features  form  the  basis  for  real-time  communications  between  the  analyst  and  user. 

Communication  skills  for  analysts  include  verbal  and  writing  skills,  but  they  can  also 
encompass  the  development  of  new  CMs,  such  as  learning  a  foreign  language. 
Foreign  language  is  of  special  importance  to  COMINT  and  HUMINT  intelligence  produc¬ 
tion  for  obvious  reasons.  From  a  general  standpoint,  language  skills  are  vital  to  the 
analyst  for  understanding  .the  meaning  of  collected  intelligence  even  after  it  is  inter¬ 
preted.  Writing  and  speaking  skills  are  essential  for  the  analyst  to  communicate 
analytic  results  to  users. 

While  the  basic  skills  of  Intelligence  analysis  are  essentially  meta-skills  that  encom¬ 
pass  intelligence  analysis  as  a  whole,  these  skills  are  used  in  the  performance  of 
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task  segments.  Task  segments  are  the  units  that  are  combined  in  the  performance  of 
analytic  tasks.  The  identification  of  those  task  segments  during  the  IMTIA  research 
lead  to  our  understanding  of  the  cognitive  processes  of  analysis  and  ''ave  since  pro¬ 
ven  Important  in  developing  training  materials  fot  tactical  and  strategic  analysts. 


6.  THE  TASK  SEGMENTS  OF  INTELLIGENCE  ANALYSIS 


Although  Intelligence  analysis  Is  a  complex,  challenging  process,  the  cognitive  skills 
used  In  analysis  are  not  very  different  from  the  skills  used  in  solving  problems  that 
arise  In  the  analyst's  personal  environment.  The  main  distinctions  are  that  the  cogni¬ 
tive  skills  of  analysis  are  more  systematically  structured  and  require  more  discipline. 

Some  of  the  more  important  analytic  skills  were  described  in  Chapter  4.  These  skills 
are  combined  to  produce  mentally-oriented  performances.  The  IMTIA  research  identi¬ 
fied  nine  such  mentally-oriented  performances,  or  generic  task  segments,  that  are 
required  for  all  intelligence  analysis  tasks. 

These  task  segments  are  used  in  varying  combinations  for  different  types  of  tasks, 
but  they  are  all  required  at  various  times  in  the  performance  of  intelligence  analysis. 
The  analyst  should  be  aware  of  these  task  segments  as  they  contribute  to  the  sys¬ 
tematic  conduct  of  analysis  since  each  segment  has  an  Identifiable  contribution  to 
the  product  of  a  goal-oriented  analytic  process.  The  task  segments  are  represented 
as  a  sequence  within  a  rotating  wheel  (see  Figure  1-1)  to  illustrate  the  conceptual 
relationship  of  each  task  segment  to  the  analyst’s  active  memory  processes.  The 
wheel  also  suggests  a  sequential  order,  but  this  is  not  strictly  the  case.  For  any 
specific  analytic  task,  individual  task  segments  may  receive  more  or  less  attention  or 
may  not  be  performed  at  all.  Some  task  segments  are  generally  performed  more  than 
once  in  the  course  of  achieving  a  product  goal. 

Within  the  intelligence  production  cycle,  task  segments  exist  between  the  time 
Interval  during  which  the  analyst  replaces  an  existing  threat  model  (see  Chapters  6 
and  7  )  with  a  new  threat  model.  One  of  the  analyst’s  goals  is  to  create  an  ideal 
p  net  that  will  reduce  the  user’s  uncertainty  with  respect  to  the  threat  model. 
This  product  requirement  cycle  is  the  framework  within  which  the  task  segments  are 
executed. 

The  nine  basic  task  segments  as  identified  in  the  IMTIA  studies  are  as  follows: 

•  Recognizing  goals  and  objectives  (always  required). 

•  Establishing  baseline  (always  required). 

•  Recognizing  uncertainties  (required  as  a  byproduct  of  several  other  task  seg¬ 
ments). 

•  Information  gathering  and  interpretation  (not  always  required). 

•  Hypothesis  formulation,  including  hypothesizing  the  threat  model  (not  always 
required). 

•  Hypothesis  testing  (not  always  required). 

•  Cataloging  analytic  procedure  and  results  (always  required). 

•  Evaluating  results  (always  required). 

•  Formulating  the  output  (always  required). 

The  specific  details  of  these  task  segments  differ  according  to  the  contents  and 
context  of  the  analytical  task  to  be  performed.  The  relationship  of  the  task 
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segments  to  the  goals,  production  requirements,  users,  and  work  setting  contexts 
was  shown  In  Figure  1-1. 


6.1  Recognizing  Goals  and  Objectives 

In  the  military  community,  goals  are  usually  called  objectives.  Objectives  tend  to  be 
more  concrete  than  goals,  but  in  either  case,  a  goal  or  an  objective  is  the  desired 
end  or  state  toward  which  effort  is  directed. 

Recognizing  a  goal  or  an  objective  means  identifying  it,  understanding  what  it  really 
means  and  the  reasons  for  it.  Examples  are: 

•  Recognizing  that  targets  have  to  be  classified  by  type  because  weaponeering 
decisions  are  based  on  target  type. 

•  Recognizing  the  need  to  know  enemy  capabilities  to  be  able  to  predict  the 
enemy’s  battlefield  effectiveness. 

•  Recognizing  that  changes  in  enemy  locations  may  signify  a  change  in  intentions. 

The  overall  aim  of  intelligence  analysis  is  to  Increase  the  user’s  ability  to  control  the 
situation  through  an  understanding  of  the  enemy  threat.  Toward  this  end,  regardless 
of  how  well  the  Incoming  intelligence  reports  have  been  prepared,  the  analyst’s 
specific  objectives  cannot  be  met  unless  the  analyst  engages  in  appropriate 
conceptually-driven  processing  activities.  Ali  analysts  must  recognize  the  work- 
setting  context  and  the  user  requirements  implicit  in  the  mission  objective.  They 
must  also  recognize  the  higher-level  goals  inherent  in  the  intelligence  community, 
since  they  provide  the  positive  values  associated  with  superior  analytic  perfor¬ 
mance. 

Several  concepts  and  assumptions  are  relevant  to  understanding  the  importance  of 
goals: 

a  Except  perhaps  for  unconditioned  and  established  conditioned  reflexes,  all 
behaviors  are  implicitly  or  explicitly  impelled  by  goals. 

•  Goals  are  defined  as  states  to  be  achieved  through  manipulation  of  one’s 
environment. 

•  Unrecognized  goals  may  lead  to  confusion  and  detours  in  achieving  those  goals. 

•  When  goals  are  recognized  and  defined,  it  is  possible  to  establish  the  optimum 
paths  to  achieve  these  goals.  These  paths  consist  of  sub-goals  and  tasks. 

•  Effectiveness  and  task  performance  is  a  function  of  how  well  goals  are  defined. 

Recognized  goals  can  be  specified  at  different  levels.  Goal  hierarchies  can  be  esta¬ 
blished  that  vary  over  several  dimensions,  such  as: 

•  Timeframe. 

•  Analyst’s  values. 

•  Importance  for  individual  survival. 

•  Importance  for  community  survival. 
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•  Concreteness  (can  they  be  objectively  observed  or  do  they  represent  abstract 

Ideals?). 

•  "Repeatability"  (are  they  ona-tlma  goals,  or  do  they  constantly  renew  them¬ 
selves?). 

•  Conflicts  with  other  goals. 

There  Is  a  hierarchy  of  goals,  rather  than  a  clear  delineation  between  goals  and 
subgoals.  The  difference  between  a  goal  and  a  subgoal  is  one  of  emphasis  and 
Importance  and  it  can  change  over  individuals  and  over  situations.  Goal  hierarchies 
can  be  used  to  determine  the  tasks  that  reed  to  be  performed  to  reach  a  desired 
state.  Goal  hierarchies  must  be  established  for  each  relevant  situation,  and  at  least 
some  of  the  goals  within  the  hierarchy  must  be  important  (have  affective  value)  to 
the  Individual.  Goals  have  different  degrees  of  importance  to  individual,  but  without 
Importance,  they  cannot  be  effective  in  influencing  behavior. 

The  difference  between  goals  and  tasks  is  that,  within  a  specified  timeframe,  goals 
tend  to  be  invariant,  whereas  tasks  can  change.  Goals  are  things  to  be  achieved; 
they  are  states  of  the  world.  Tasks  are  the  means  for  achieving  goals  and  they  can 
change  In  that,  for  any  given  goal,  there  may  be  several  ways  to  reach  it.  Tasks  are 
effective  to  the  extent  that  they  represent  means  to  achieve  a  goal.  Thus,  if  the 
end  product  of  a  task  Is  unsatisfactory  with  respect  to  the  goal,  one  changes  the 
task,  not  the  goal. 

Tasks  are  concrete  events  that  are  pre-planned  to  achieve  desired  goals  or 
subgoals  and  performance  Is  the  implementation  of  tasks.  Tasks  Involve  changing 
certain  aspects  of  one’s  environment  so  as  to  reach  the  goal. 

Having  clear  and  well-defined  goals  facilitates  the  task  of  the  analyst  in  several 
Important  ways: 

•  It  provides  for  ordered  knowledge  usage  through  active  memory  since  goals  are 
integrated  within  each  conceptual  model. 

•  It  allows  for  optimum  sequencing  of  \asks,  subtasks,  and  task  segments. 

•  It  facilitates  the  development  of  effective  strategies. 

•  It  leads  to  effective  tradeoffs  between  time  resources  and  other  activities. 

•  It  facilitates  the  resumption  of  processing  after  an  interruption. 

•  It  provides  the  basis  for  evaluation  of  performance  and  achievement. 

•  It  makes  It  possible  to  evaluate  the  completion  of  actions. 

•  It  saves  time  by  focusing  the  analysis  on  actual  information  needs. 

The  use  of  goals  in  learning  from  complex  materials  has  demonstrated  convincingly 
that  learning  goals  Induce  the  learner  to  process  the  material  in  such  a  way  that  per¬ 
formance  on  test  questions  (usually  sentence  completion  items)  referring  to  the 
goal-relevant  material  13  improved.  This  improvement  cannot  be  explained  solely  as  a 
redistribution  of  processing  time.  The  extent  of  the  improvement  is  somewhat 
dependent  upon  the  number  of  goals  to  be  mastered  and  the  ease  with  which  the 
learner  can  locate  the  appropriate  material  In  a  text.  With  a  greater  number  of 
goals,  most  subjects  take  longer  to  study  the  materia!  and  they  are  less  likely  to 
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learn  the  Information  relevant  to  each  goal.  If  all  of  the  data  that  are  relevant  to  a 
particular  goal  are  not  located  together  In  the  Information  flow,  then  it  will  sometimes 
be  the  case  that  only  the  Information  contained  In  the  first  reference  to  the  goal¬ 
relevant  data  will  be  thoroughly  studied.  Iherefore,  there  are  some  limiting  factors  in 
adopting  learning  goals  as  learning  guides,  and  the  limits  are  dependent  upon  both 
the  learner  and  the  materials.  The  available  basic  research  suggests  that  each 
analyst  should  (a)  uoopt  only  a  limited  number  of  goals  to  guide  performance  and  (b) 
acknowledge  potential  Interpretive  biases  caused  by  concentrating  too  heavily  on 
the  Initial  Information  pertinent  to  the  goals.  Selectivity  biases  relevant  to  goai- 
directed  performance  are  described  in  Section  5.2.1  and  in  Section  10.4. 


6.2  Shared  Goals 

In  developing  shared  conceptual  models  and  in  identifying  relevant  goals,  it  is  impor¬ 
tant  to  make  a  connection  between  the  goals  that  are  to  be  shared  and  goals  that 
are  already  important  to  the  individual.  For  example,  according  to  Godson,  there  is 
much  competition  and  (in-house  politics)  in  the  Intelligence  community.  The  goals  of 
the  Individual  analysts  tend  to  be  (selfish)  ...  analysts  are  not  rewarded  for  "good 
analysis"  ...  there  is  no  positive  feedback  for  having  contributed  to  the  shared  goal... 

Thus,  both  during  training  and  in  the  work  setting,  it  is  important  to  consider  the  fol¬ 
lowing: 

Make  the  goals  of  analysis  personally  important  to  each  analyst 

Individuals  have  different  goals,  but  for  teamwork  to  be  effective,  there  should  be 
some  shared  goals  at  some  level  of  the  hierarchy. 

There  Is  an  hierarchy  of  goals,  but  all  of  them  are  Important.  All  tasks  have  implicit 
goals  associated  with  them,  and  most  human  behavior  is  guided  by  goals  whether 
Implicitly  or  explicitly.  However,  goals  must  be  clearly  defined  (i.e.,  they  must  be 
spelled  out  during  training)  and  they  must  have  some  "affective  value"  to  be  effec¬ 
tive  In  facilitating  learning,  retention,  recall,  and  performance  ot  a  task. 

Shared  goals  promote  the  development  of  SCMs.  However,  analysts  need  not  accept 
the  users'  goals  as  their  own  to  produce  the  ideal  product  for  the  user.  Analysts  do 
need  to  know  and  understand  the  users’  goal  structure,  though. 

In  conclusion,  for  purposes  of  training  and  Improvement  of  analytic  performance,  it  is 
thus  necessary  to  identify  a  taxonomy  of  goals,  subgoals,  and  tasks  and  to  relate 
this  taxonomy  to  the  cognitive  skills  required  to  perform  the  task. 

The  Identification  of  cognitive  skills  in  Section  8  is  the  result  of  combining  considera¬ 
tions  from  the  cognitive  model  with  the  underlying  requirements  for  analytic  tasks. 


6.3  Establishing  Baseline 

Baselines  are  the  Initial  conditions  at  the  start  of  the  analysis  task,  along  with  the 
knowledge  currently  available  that  is  relevant  to  it. 

For  example,  a  baseline  threat  model  is  a  representation  of  a  limited  sec'ioi  of  the 
real  world  that  describes  currently  known  events  and  conditions  comprising: 
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•  Enemy  forces,  behavior,  and  installations. 

•  Friendly  forces,  behavior,  and  installations. 

•  Environment  (geography,  constructions,  weather,  populations,  etc.). 

A  base//ne  threat  model  is  what  Is  known  about  those  events  and  conditions  at  the 
moment.  Establishing  a  baseline  threat  model  related  to  friendly  objectives  consists 
of: 

o  Recognizing  the  events  and  conditions,  described  in  the  threat  model,  that  might 
affect  friendly  objectives 

•  Filtering  out  the  events  and  conditions  that  are  unlikely  to  affect  suen  objec¬ 
tives. 

•  Integrating  the  selected  events  and  conditions  into  a  modal  of  the  battlefield 
that  la  unified  and  restricted  by  the  objectives. 

•  Understanding  the  historical  origins  of  the  threat  and  the  events  leading  to  the 
conflict  situation. 


5.4  Formulating  Hypotheses 

Hypothesis  formulation  is  the  task  whereby  plausible  visualizations  about  the  teal 
world  are  created  in  the  context  of  the  mission  objective  and  threat  model. 

The  possible  outcomes  of  the  various  combinations  of  potential  real-world  events 
that  can  occur  must  be  imagined,  in  order  to: 

•  Predict  those  events  that  are  most  likely  to  occur. 

•  Assess  the  impact  of  those  events  on  the  task  objective'  ;). 

The  Importance  of  formulating  hypotheses  is  most  obvious  in  the  process  of  updating 
the  threat  model,  during  which  the  enemy's  probable  objective(s)  must  be  discrim¬ 
inated  from  the  universe  of  his  possible  courses  of  action.  For  the  sake  of  clarity, 
much  of  this  genera  description  references  this  task  context. 

In  order  to  describe  how  to  formulate  a  hypothesis,  it  is  essential  to  understand  what 
a  hypothesis  is  and  how  it  is  constructed.  The  relationships  that  exist  between  dif¬ 
ferent  kinds  of  events  that  take  place  in  the  world  can  be  classified  in  terms  of  their 
geniality  and  logical  role.  In  following  sections,  it  W'!l  become  clear  that  each  kind 
of  event  is  associated  with  particular  types  of  uncertainty  and  collection/  interpre¬ 
tation  problems. 

It  is  sometimes  assumed  that  intelligence  analysis  activities  are  driven  mostly  by 
data.  This  view  implies  that  the  objective  of  reducing  battlefield  uncertainty  is 
reached  by  examining  all  of  the  relevant  information  about  the  battlefield— SIGINT 
reports,  IMINT  data,  etc.  This  view  assumes  that  a  model  can  be  created  of  what  is 
actually  taking  place  and  of  what  will  take  place  in  the  future.  In  truth,  the  amount 
of  raw  data  generated  on  the  modern  battlefield  is  too  enormous  for  a  single  analyst, 
or  even  a  team  of  analysts,  to  fully  process.  Even  If  all  the  data  could  be  assessed, 
the  number  of  possibilities  that  the  information  implies  would  also  be  too  vast  to 
evaluate. 
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In  reality,  analysts  speculate  on  what  possibilities  are  likely  or  important,  given  the 
baseline  model  that  pertains  to  the  current  task.  These  possibilities,  termed  planned 
outcomes,  are  a  subset  of  the  universe  of  possible  outcomes.  By  selecting  a  likely 
subset,  analysts  can  concentrate  on  assessing  the  likelihood  of  a  reasonable  number 
of  cases.  This  Is  what  Is  meant  by  the  notion  that  Intelligence  analysis  is  as  much 
"goal-driven"  as  "data  driven."  The  driving  force  is  the  task  objective  (that  is,  what 
la  required  to  reduce  uncertainty?),  and  not  the  chaotic  collection  of  data  and  noise 
that  abounds  on  the  battlefield.  The  data  obviously  have  a  major  role,  but  as  will  be 
shown,  the  role  involves  Interpretation  and  evaluation. 

To  determine  the  likelihood  of  a  planned  outcome  of  such  a  selected  subset  of  data, 
the  pattern  of  activities  that  lead  up  to  the  event,  that  is,  patterns  that  predict  it, 
must  be  recognized.  Individual  activities  are  sometimes  called  objective-oriented 
processes.  This  is  because  they  exist  to  serve  a  specific  objective,  and  are 
dynamic  (change  over  time).  Specific  objective-oriented  processes  are  related  in 
different  ways  to  a  given  planned  outcome.  The  ways  they  are  related  can  often  be 
expressed  using  operators  like  "AND,"  "OR,"  and  "NOT."  For  example,  movement  of 
enemy  troops  toward  the  FEBA,  OR  movement  of  materiel  toward  the  FEBA,  AND  mass¬ 
ing  of  enemy  troops  in  echelon  may  be  processes  associated  with  an  eminent  attack 
(the  planned  outcome). 

The  planned  outcome  and  Its  related  objective-oriented  processes  are  the  real  world 
"Ingredients"  that  form  the  basis  of  a  hypothesis.  When  converted  into  the  language 
of  logic,  the  objective-oriented  processes  are  known  as  propositions,  and  the 
olanned  outcome  is  simply  referred  to  as  the  outcome.  A  set  of  propositions  that  are 
logically  related  (through  vne  use  of  operators),  and  that  imply  an  outcome,  comprise 
a  hypothesis.  The  hypothesis  states  that  If  the  logical  relationship  among  the  propo¬ 
sitions  Is  satisfied,  the  outcome  will  occur. 

In  the  threat  model  (see  Chapters  6  and  7)  for  example,  a  manageable  number  of 
possible  enemy  courses  of  action  must  be  described.  For  each  of  the  enemy’s 
potential  objectives,  a  set  of  propositions  are  collected  that  predict  the  associated 
cbjoctive.  The  propositions  will  relate  to  the  enemy  forces,  the  environment,  and  the 
mission  of  the  friendly  forces.  Essentially,  the  analyst  thinks  through  the  enemy’s 
war  plan  In  order  to  describe  those  processes  that  must  occur  before  a  given  objec¬ 
tive  is  obtained.  This  thinking  process  is  based  on  the  analyst’s  knowledge  of  enemy 
doctrine  and  experience,  and  the  collected  experience  of  the  intelligence  community 
and  results  in  determining  what  evidence  is  available  or  needs  to  be  collected  to 
accept  or  reject  the  propositions. 

The  degree  of  certainty  associated  with  the  evidence,  together  with  the  degree  of 
belief  in  the  hypothesis  itself  (that  is,  given  that  the  logical  relationship  between  the 
propositions  is  satisfied,  how  likely  is  it  that  the  outcome  is  actually  implied?)  deter¬ 
mines  how  strongly  the  analyst  believes  that  the  outcome  will  take  place.  The  evi¬ 
dence  used  to  assess  the  truth  of  the  propositions  must  be  based  on  real-world 
data. 

These  data  will  be  In  the  form  of  reports  about  potentially  observable  events,  that  is, 
events  that  can  be  observed  directly.  Such  events  are  "snapshots"  in  time;  they 
represent  the  status  of  an  objective-oriented  process  at  a  given  moment.  Multiple 
snapshots  allow  the  course  of  a  process  to  be  determined,  permitting  the  process  to 
be  characterized  with  greater  certainty.  An  objective-oriented  process  is  basically 
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a  logical  concept — It  is  a  construct  created  to  relate  a  hypothetical  outcome  to 
observable  events.  Often  based  on  enemy  doctrine,  it  answers  the  question  "why" 
particular  observed  events  are  taking  place. 

When  applied  to  confirming  or  dlsconfirming  propositions,  such  events  are  known  as 
Indicators.  An  Indicator  of  troop  movement  toward  the  FEBA  might  be  a  large  number 
of  loaded  trucks  headed  west  on  a  major  route.  The  same  indicator  might  apply  to  a 
proposition  concerning  movement  of  materiel.  Because  a  single  indicator  may  confirm 
(or  disconfirm)  more  than  one  proposition,  multiple  indicators  must  often  be  collected 
against,  in  order  to  assess  the  truth  of  a  single  proposition  or  to  discriminate  among 
several  propositions. 

Indicators  represent  the  potentially  observable  events  that  form  the  basis  of  collec¬ 
tion  tasks.  These  events  are  not  usually  observed  directly.  Instead,  analysts  rely 
on  the  signatures  of  these  events  to  assess  the  truth  of  propositions.  Signatures 
are  measurable  events  that  take  place  In  the  electromagnetic  spectrum.  This  con¬ 
cept  must  be  Interpreted  broadly;  signatures  could  be  SIGINT  reports,  or  the  report 
of  a  human  eye-witness  to  an  event  (based,  or  course,  on  the  light  reaching  the  eye 
of  the  witness).  The  signature  is.  Itself,  devoid  of  meaning.  When  It  is  Interpreted 
as  an  Indicator,  it’s  meaning  is  derived  from  the  mission  context  and  from  the  threat 
model. 

As  can  be  seen  from  this  description,  analysis  takes  place  from  the  "top  down," 
starting  with  planned  objectives  that  are  potentialities.  A  model  Is  the  created  that 
decomposes  a  planned  objective  Into  its  dynamic  processes.  These  processes  are 
then  assessed  In  terms  of  Indicators  against  which  actual  data  are  collected.  This 
permits  the  testing  of  a  hypothesis  that  a  given  outcome  will  take  place  (or  is 
planned). 


6.5  Recognizing  Uncertainties 

The  hypothesized  threat  model  will  always  have  shortfalls  in  providing  a  true 
representation  of  the  battlefield  and  the  outcomo.  Analysts  must  recognize  where 
uncertainties  exist  in  the  threat  model,  evaluate  the  importance  of  those  uncertain¬ 
ties  to  the  mission,  and  focus  information  collection  activities  on  those  areas  of  signi¬ 
ficance. 

As  part  of  the  process  of  recognizing  uncertainties,  the  following  questions  can  be 
asked: 

•  Is  the  threat  model  adequate? 

—  Have  all  of  the  plausible  options  the  enemy  might  Include  in  his  war  plan 
been  included? 

—  Have  all  relevant  aspects  of  the  environment  been  considered? 

—  Are  the  user's  mission  objectives  known? 

•  la  there  an  encompassing  list  of  information  items  to  be  provided  on  the  threat 
model? 

a  Are  the  Information  requirements  covered  appropriately  with  a  collection  plan? 
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•  Are  the  required  resources  to  gather  and  interpret  collected  information  avail¬ 
able? 

•  Is  there  a  plan  for  utilizing  new  Information  in  the  threat  model? 

•  Is  there  a  plan  for  communicating  the  threat  model  Information  to  the  user? 

•  What  Is  the  reliability  of  the  collection  systems?  (Unreliable  collection  systems 
create  uncertainties  about  the  collected  information  to  be  used  as  evidence  in 
testing  hypotheses.) 

•  Are  there  differences  between  past  and  present  collection  plans?  (The  differ¬ 
ence  between  past  and  present  collection  plans  can  indicate  uncertainties  con¬ 
cerning  missing  information.) 

•  What  are  the  effects  of  the  environment  on  the  collection  process?  (Noise, 
errors,  and  communication  gaps  create  uncertainties.) 

•  How  Is  the  information  to  be  used?  (Knowing  this  information  reduces  uncertain¬ 
ties  with  respect  to  the  utility  of  the  product.) 

•  What  is  the  reliability  of  the  analysis?  (Unreliable  analyses  create  uncertainties 
about  the  validity  of  interpretation.) 

5.6  Gathering  and  Interpreting  Information 
Gathering  information  consists  of: 

•  Assembling  already  available  information. 

•  Generating  requests  for  Information  or  tasking  orders  for  collection  systems. 

•  Receiving  the  information  from  sources  via  communications  means. 

Interpreting  information  consists  of: 

•  Recognizing  that  information  fits  known  or  expected  patterns. 

•  Recognizing  that  informatic  .  is  different  from  known  or  expected  patterns, 

•  Establishing  belief  about  the  truth  of  the  information. 

•  Establishing  the  utility  of  the  information  for  accomplishing  an  objective. 

•  Integrating  the  information  into  the  structure  of  the  threat  model. 

5.7  Testing  Hypotheses 

Testing  hypotheses  is  the  task  segment  that  leads  to  accepting  or  rejecting  a 
hypothesis.  Hypotheses  can  also  be  deferred  for  future  testing  based  on  the  out¬ 
come  of  an  initial  test. 

As  described  earlier,  a  hypothesis  is  composed  of  a  set  of  logically  related  proposi¬ 
tions  and  an  outcome.  The  propositions  represent  processes  presumed  to  be  predic¬ 
tive  of  a  planned  outcome.  These  processes  are  assessed  by  reference  to  evidence 
In  the  form  of  observable  events  or  Indicators.  The  indicators  are  essentially 
"snapshots"  of  the  process  arid  are  signified  by  electromagnetic  signatures 
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processed  by  the  collection  system. 

Although  a  hypothesis  can  be  rejected  on  the  basis  of  disconfirming  evidence,  it  is 
not  possible,  In  formal  logic,  to  "prove"  a  hypothesis.  The  most  that  can  be  done  is 
to  accept  a  hypothesis  on  the  grounds  that  the  evidence  supports  it  better  than  any 
rival  hypothesis.  Generally,  the  concept  of  "degree  of  belief"  is  a  stronger  one  to 
apply  In  hypothesis  testing  than  simple  acceptance  or  rejection. 

The  degree  of  belief  that  can  be  assigned  to  a  hypothesis  depends  on  the  degree  to 
which  the  evidence  supports  the  propositions  and  the  degree  to  which  the  logical 
combination  of  the  propositions  truly  implies  the  outcome.  Both  of  these  areas 
involve  uncertainties  that  must  be  recognized.  Consider  the  following: 

•  Sensor  systems  must  detect  signatures  in  a  noisy  atmosphere;  there  is  predict¬ 
able  uncertainty  in  distinguishing  between  signals  and  noise. 

•  You  cannot  be  absolutely  certain  that  the  signatures  imply  an  indicator. 

•  You  cannot  be  absolutely  certain  that  the  indicator  implies  the  proposition. 

•  You  cannot  be  absolutely  certain  that  the  propositions  Imply  the  outcome. 

The  element  of  deception  enters  into  the  hypothesis  testing  process  by  producing 
false  or  misleading  signatures  and  indicators.  In  preparation  for  a  course  of  action, 
the  enemy  can  select  options  that  are  not  optimum,  but  will  suffice  for  the  planned 
action.  The  indicator  will  therefore  point  most  convincingly  to  an  erroneous 
hypothesis,  while  the  actual  outcome  appears  to  be  iess  strongly  supported. 
Analysts  may  be  misled  into  accepting  a  hypothesis  for  which  the  indicators  support 
the  optimum  objective-oriented  processes. 

Two  concepts  are  of  particular  importance  In  dealing  with  uncertainty  in  the  testing 
of  hypotheses.  The  first  is  reliability.  This  term  normally  refers  to  the  degree  of 
repeatability  of  a  measurement.  (That  Is,  given  several  repetitions  of  the  same  sig¬ 
nature,  how  consistently  would  a  collection  system  report  the  same  measurements?) 
Reliability  can  also  be  applied  to  logical  propositions.  (That  Is,  how  often  would  con¬ 
firmation  of  the  propositions  of  a  hypothesis  be  followed  by  the  expected  outcome?) 

The  second  concept  is  validity.  A  signature  is  valid  if  its  meaning  is  correctly 
assessed,  that  is,  if  it  is  in  fact  associated  with  the  designated  indicator.  An  indica¬ 
tor  13  valid  if  it  truly  indicates  the  proposition  with  which  it  Is  associated. 

Improper  Judgment  of  these  factors  can  result  in  two  types  of  error.  The  first  is  the 
probability  of  falsely  rejecting  a  hypothesis.  The  second  is  the  probability  that  a 
hypothesis  has  been  falsely  accepted  (a  "false  alarm").  Since  analysts  usually  con¬ 
sider  a  set  of  hypotheses  and  attempt  to  reduce  it  to  either  one  or  a  smaller  set 
(depending  on  the  user’s  requirements),  both  of  these  errors  are  usually  committed 
together. 

In  addition  to  carefully  assessing  the  reliability  and  validity  of  the  evidence  that  sup¬ 
ports  or  dlsconfirms  hypotheses,  It  is  necessary  to  pay  special  attention  to  how  criti¬ 
cal  a  piece  of  evidence  is,  how  vulnerable  to  deception  it  is,  and  iiow  well  it  discrim¬ 
inates  among  the  competing  hypotheses.  Not  all  evidence  need  be  applied  to  the 
test  with  equal  weight.  Evidence  that  does  not  discriminate  between  hypotheses  is 
not  useful.  More  reliable  evidence  can  be  weighted  more  heavily.  Evidence  that  is 
subject  to  deception  must  be  applied  cautiously,  and  with  sensitivity  to  evidence 
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that  might,  itself,  permit  a  test  of  the  hypothesis  that  deception  is  being  practiced. 

If  a  decision  cannot  be  made  on  the  basis  of  the  available  evidence  (that  is,  if  the 
degree  of  belief  among  competing  hypotheses  is  close),  final  evaluation  of  the  evi¬ 
dence  may  be  deferred  until  more  data  become  available,  3efore  making  this  deci¬ 
sion,  however,  the  user’s  Information  requirements  and  the  time  frame  of  the  user’s 
decisions  must  be  carefully  considered. 

6.3  Evaluating  Results 

Analysis  is  an  Itu  .  -  ~s  that  could  continue  indefinitely.  Evaluating  results  is 

a  task  segment  i,-.  .oSi-ury  tor  knowing  how  long  the  analytic  process  should  be  pur¬ 
sued  and  for  evaluating  productivity.  Knowing  when  to  stop  can  be  determined  by 
asking  the  fo>.  wing  questions: 

•  Have  all  mission  information  requirements  been  identified? 

•  Is  there  an  adequate  collection  plan? 

•  Has  the  collection  plan  been  executed  properly? 

•  Has  the  threat  model  been  updated? 

•  Has  the  product  been  prepared? 

•  Has  the  product  beer  communicated? 

Each  question  reflects  an  iterative  task  of  reducing  uncertainties.  It  is  unlikely  that 
all  uncertainties  will  ever  be  reduced  to  a  point  of  having  "ground  truth."  Since 
results  will  never  be  perfect,  the  question  of  "When  am  I  done?"  is  determined  by 
when  time  and  information  resources  are  exhausted. 

Analysts  must  learn  to  prioritize  the  order  in  which  tasks  are  accomplished  to  maxim¬ 
ize  the  effectiveness  of  the  resources.  In  all  cases,  this  ordering  is  based  on  judg¬ 
ing  huw  accomplishing  the  Various  tasks  relates  to  reuuciny  tire  risk  tu  Lite  mission. 

6.9  Formulating  Output 

Output  is  the  intelligence  product.  Formulating  output  means  to  "tailor"  it  for  a  par¬ 
ticular  user’s  benefit. 

To  insure  that  the  output  is  adapted  to  the  user’s  needs,  analysts  should  ask  them¬ 
selves: 

e  Do  I  know  and  understand  the  user's  objectives? 

•  What  are  the  constraints  under  which  the  user  is  currently  working? 

•  Do  i  know  enough  about  the  way  the  user  thinks  so  that  I  can  communicate  the 
product  appropriately? 


6.10  Cataloging 

Cataloging  Is  the  task  segment  used  in  recording  significant  Information  about  the 
analytic  process  Itself  and  in  transferring  that  information  to  appropriate  users 
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and/or  to  storage.  Cataloging  requires  extreme  discipline  and  a  systematic 
approach.  Tne  recorded  Information  that  results  from  cataloging  is  the  primary  basis 
for  communication  within  the  Intelligence  organization  and  with  intelligence  users  It 
also  permits  retrieval  of  information  for  use  In  future  analyses. 

In  determining  what  items  to  catalog,  analysts  should  ask  questions  such  as: 

•  How  crucial  Is  It  to  remember  this  information? 

•  Can  I  use  this  information  again  at  a  later  time  when  I  might  have  difficulty 
remembering  it? 

•  Is  someone  else  going  to  benefit  from  the  recording  of  this  information? 

•  Is  this  Information  necessary  to  justify  the  intelligence  product? 

•  How  frequently  is  this  item  refreshed  or  replaced  by  new  collection  activities.  " 

•  How  Important  are  trends  or  variability  in  this  item? 

•  Might  I  have  to  reinterpret  this  Information  at  a  later  time? 

Any  of  the  tasks  performed  In  analysis  may  require  cataloging  significant  items  for 
later  use. 
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6.  THE  CONCEPT  OF  THREAT 


Many  aspects  of  Intelligence  analysis  are  due  to  the  fact  that  "Threat"  is  the  target 
of  analysis.  Threat  Is  a  complex  concept  that  involves  context  (threatener,  reci¬ 
pient,  time,  geographic  location);  communication  of  threat,  perception  of  threat  (by 
analyst,  by  user);  and  control  of  threat  (physical  and  cognitive).  Threat  is  an  antici¬ 
pated  danger  to  a  National  Interest  in  a  particular  contingency  from  a  foreign  force. 

Threat  perceptions  and  the  reactions  to  the  various  factors  of  threat  are  individual¬ 
istic,  but  threat  must  be  treated  in  a  standardized  manner  because  of  the  complexity 
of  the  various  factors  that  underlie  threat  pe  ev^n  and  the  potential  seriousness 
of  threats  against  the  national  interests. 

clysts,  among  each  other  and  with  the  us-n  .  intelligence,  need  to  develop  the 
san, a  understanding  of  the  battlefield  co  ext.  Tn  purpose  of  modeling  the  threat  is 
to  standardize  these  perceptions;  the  purpose  of  verbalizing  how  people  think,  react, 
analyze,  and  Interpret  threat  Is  to  lay  the  groundwork  for  developing  a  shared  con¬ 
ceptual  model  of  threat. 


0.1  Perception  of  Threat 

The  various  aspects  associated  with  the  perception  of  threat  have  an  important 
Influence  on  how  threat  is  controlled,  both  by  the  analyst  and  the  user.  Although  the 
analyst’s  perception  of  threat  is  frequently  mitigated  by  factors  extraneous  to  the 
dictates  of  good  analysis,  such  as  national  policy,  organizational  characteristics,  user 
Idiosyncrasies,  or  simply  "conventional  wisdom"  (a  mild  form  of  prejudice,  according 
to  Codevilla,  1980),  using  a  common  threat  model  can  do  much  to  alleviate  some  of 
these  extraneous  problems. 

The  five  aspects  most  relevant  to  the  perception  of  threat  are  described  beiow. 


6.7.7  ASSESSING  THREAT 

Analysts  use  numerous  specific  analytical  skills  in  assessing  threat.  Threat  is  a 
multi-dimensional  situation  that  involves  many  uncertainties  and  many  contingencies. 
In  assessing  threat,  the  analyst  must  have  a  good  understanding  of,  knowledge  of, 
and  ability  to  use  problem  structuring  and  decision  making  techniques,  as  well  as 
techniques  for  evaluating  and  dealing  with  uncertainty.  These  techniques  must  be 
combined  with  a  thorough  knowledge  of  the  environment  and  of  the  available 
resources. 


6.7.2  THREAT  CREDIBILITY 

The  credibility  of  a  threat  is  a  function  of  intention  and  capability  on  the  part  of  the 
threatener.  The  credibility  of  an  intention  is  difficult  to  evaluate  since  it  involves 
anticipating  how  the  enemy  thinks.  An  enemy  force’s  capability,  however,  can  be 
evaluated  In  the  more  concrete  terms  of  weapon  performance  and  lethality,  force 
alze,  disposition  of  forces,  combat  effectiveness,  and  other  factors.  Predicting  the 
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capability  of  a  future  force  involves  additional  factors  such  as  estimates  of  the 
nation’s  manpower  and  industrial  base,  technology,  training  capabilities,  supply  lines, 
and  alliances. 


6.1.3  COMMUNICATING  THREAT 

To  communicate  threat,  there  must  be  a  shared  conceptual  model  of  the  threat 
between  the  analyst  and  the  user.  The  importance  of  shared  conceptual  models  in 
communication  was  discussed  In  Chapter  3.  Accurate  communication  about  the  threat 
is  possibly  the  mos.  important  aspect  of  the  analyst-user  relationship,  since  an  accu¬ 
rate  understanding  of  threat  is  basic  to  making  correct  decisions  concerning  the  bat¬ 
tlefield  situation.  Training  the  analyst  to  understand  how  the  users  of  intelligence 
think,  and  what  their  requirements  are,  is  a  prerequisite  to  the  establishment  of  a 
common  model  of  threat  between  analyst  and  user. 


6.1.4  REACTION  TO  THREAT 

The  user's  reaction  to  the  threat  will  depend  on  perception  of  danger  and  the  degree 
of  control  over  the  situation.  The  level  of  danger  represented  by  the  threat  is  the 
potential  for  harm,  physical  loss,  or  injury,  if  the  threat  is  carried  out.  The  danger 
represented  by  the  enemy  is  the  potential  loss  to  the  friendly  forces  in  terms  of  per¬ 
se  nel,  materiel,  or  other  national  Interests. 

Danger  is  differentiated  from  risk.  Risks  are  "taken"  when  danger  is  known  to  exist. 
Thus,  risk  is  a  voluntary  exposure  to  danger,  however  unavoidable  it  might  be  given 
the  circumstances.  In  combat,  there  is  risk  In  not  preparing  counters  to  an  enemy’s 
threat,  whether  or  not  that  threat  has  credibility.  Risk  is  increased  as  a  function  of 
potential  danger  and  decreased  when  the  enemy  threat  credibility  is  lessened. 

Risk  taking  is  a  psychological  characteristic  that  depends  on  how  people  perceive 
threat  and  react  to  it.  Milburn  and  Watman  (1981)  provide  a  view  of  threat  percep¬ 
tion  that  has  implications  for  understanding  peoples’  reactions  to  threat.  Milburn  and 
Watman’s  model  is  shown  in  Figure  6-1;  it  proposes  that  behavioral  responses  to 
threat  perception  are  a  joint  function  of  the  amount  of  danger  present  and  the 
amount  of  available  control  over  the  situation. 

The  model  further  suggests  that,  regardless  of  the  amount  of  danger  involved,  an 
increase  in  the  amount  of  control  available  will  have  a  positive  behavioral  outcome 
(sense  of  comfort  or  challenge).  One  of  the  analyst’s  most  important  goals,  there¬ 
fore,  is  to  provide  an  increase  in  the  user's  ability  to  control  the  situation.  An 
Increase  in  the  user’s  ability  to  control  the  situation  is  automatically  associated  with 
a  decrease  in  uncertainty,  thereby  fulfilling  one  of  the  goals  of  intelligence  analysis. 

The  user’s  ability  to  control  the  situation  is  very  much  dependent  on  the  analytic  pro¬ 
duct.  The  shared  threat  model  is  a  tool  used  by  both  analysts  and  users  to  ensure 
as  complete  and  accurate  a  representation  of  the  threat  situation  as  possible, 
tailored  to  the  particular  mission  requirement. 

A  good  threat  model  can  provide  both  cognitive  and  physical  control.  Having 
knowledge  of  all  the  relevant  elements  (and  the  relationship  between  the  elements) 
associated  with  a  mission,  generates  the  necessary  understanding  of  the  availability 
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and  potential  for  physical  control.  On  the  other  hand,  the  users’  knowledge  of  having 
a  shared  threat  model  with  the  intelligence  analysts,  generates  the  best  possible 

cognitive  control. 


6.1. 5  EVALUATING  THREAT 

The  evaluation  of  threat  as  the  target  of  analysis  must  always  be  performed  within  a 
particular  context  and  must  Include  the  use  of  a  "Threat  Model".  Threat  modeling 
requires  an  understanding  of  both  cognitive  and  physical  factors  that  impact  on 
analysis.  Cognitive  factors  are  those  that  determine  how  the  analyst  or  user  will 
react  to  their  perception  of  threat,  as  discussed  by  Milburn  and  Watman  (1981). 
Physical  factors  are  those  that  concern  the  dimensions  of  the  battlefield.  Both  the 
cognitive  and  the  physical  factors  underlying  perception  of  threat  depend  on  the 
amount  of  uncertainty  present  in  the  threat  situation. 

Uncertainty  plays  a  key  role  In  the  valuation  of  threat.  First,  there  is  the  uncer¬ 
tainty  about  the  enemy  Intentions  concerning  the  real  objective  of  the  threat  and  the 
enemy’s  options  for  achieving  that  objective.  Second,  there  Is  uncertainty  in 
evaluating  the  capabilities  of  the  enemy  force  to  carry  out  a  threat.  Third,  there  is 
uncertainty  in  the  five  battlefield  risk  factors,  namely,  lethality,  warning  time,  enemy 
options,  friendly  options,  and  environmental  conditions.  Fourth,  there  is  uncertainly 
that  the  method  of  response  will  produce  a  desirable  outcome. 

Thera  uncertainties  are  clearly  identified  in  the  threat  model,  thereby  minimizing  the 
lack  of  control  that  arises  from  having  uncertainties  about  uncertainties. 

In  developing  a  cognitive  model  of  the  intelligence  analyst,  and  in  identifying  the 
skills  required  to  optimize  analytic  performance,  the  IMTIA  research  laid  the  ground¬ 
work  for  Identifying  the  various  aspects  and  elements  of  the  threat  model,  end  for 
specifying  means  to  develop  it.  The  elements  of  the  threat  model  are  described  in 
the  following  section. 


6.2  Threat  Modeling 

The  intelligence  analyst  engages  in  a  complex  mental  activity  that  is  described  as 
Threat  Modeling. 

Modeling  Is  a  fundamental  intellectual  process  of  intelligence  analysis.  The  frame  of 
reference  of  the  modeling  activity  is  the  battlefield.  The  objective  of  the  modeling 
activity  Is  to  provide  representations  of  battlefield  threats  and  opportunities  for  con¬ 
trol  so  that  information  that  reduces  outcome  uncertainties  can  be  provided  to  com¬ 
manders. 

The  intelligence  analyst  is  only  one  of  the  battlefield  modelers  supporting  the  combat 
force  commander.  Other  analysts  provide  models  of  maneuver  operations,  logistics, 
combat,  and  support  missions.  Differential  interpretations  must  be  made  by  each  of 
the  operational  elements  of  a  combat  force.  The  intelligence  analyst  focuses  on 
threat  posed  by  enemy  forces  but  integrates  the  meaning  of  threat  in  terms  of  the 
significance  to  specific  national  interests  or  the  conduct  of  specific  friendly  force 
operations. 
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8.3  Features  of  the  Threat  Model 


Regardless  of  the  interpretation  of  the  threat  model,  its  basic  features  are  common 
to  all  users.  These  basic  features  comprise  three  different  perspectives,  each  hav¬ 
ing  numerous  associated  elements  that  Interact  across  perspectives. 

The  three  perspectives  of  the  threat  model  are  called  RED,  WHITE,  and  BLUE,  refer¬ 
ring  respectively  to  the  enemy,  the  environment,  and  the  friendly  forces. 


6.3.7  THE  WHITE  PERSPECTIVE 

The  initial  step  in  threat  modeling  is  always  the  development  of  the  WHITE  framework 
in  the  two  dimensions  of  space  and  time.  It  Is  the  white  framework  that  anchors  the 
threat  model  to  the  real  world. 

Developing  the  geographic  framework  of  the  battlefield  can  be  as  simple  as  plotting 
the  battlefield  on  a  map.  The  map  is  a  model  of  the  real  world.  The  topographic  map 
contains  a  wealth  of  information  about  the  battlefield  environment,  including  descrip¬ 
tions  of  terrain,  roads,  population  centers,  etc.  When  highly  detailed  military  maps 
are  available,  they  contain  much  of  the  information  that  contributes  to  the  threat 
model.  An  important  task  of  strategic  analysts  is  the  development  of  white  frame¬ 
works  in  areas  of  anticipated  threat  for  later  use  by  tactical  analysts. 

Time,  as  the  second  dimension  of  the  white  framework,  is  treated  as  a  sequence  of 
time  period  snapshots  (or  windows)  within  which  events  occur.  Threat  model  time 
can  be  compressed  to  bring  events  closer  together  or  expanded  to  make  individual 
events  more  distinguishable.  Time  windows  can  be  overlaid  on  each  other  in  order  to 
see  patterns  of  events  or  to  distinguish  changes. 

One  of  the  snapshots  in  the  threat  model  sequence  must  represent  the  current  time 
frame.  A  sequence  of  snapshots  allows  courses  of  action  to  be  followed,  from  the 
current  timeframe  to  the  eventual  outcome  of  the  battle.  Each  snapshot  can  be 
thought  of  as  an  overlay  of  information  over  the  battlefield  geography. 

Future  automated  data-processing  capabilities  might  provide  an  interactive  computer 
graphics  terminal  for  creating  the  threat  model,  carrying  static  information  from  one 
snapshot  to  the  next,  and  integrating  new  information  into  the  model. 


6.3.2  THINKING  WHITE 

Filling  in  the  time  and  space  framework  of  the  threat  model  is  the  process  of  thinking 
WHITE.  Thinking  white  creates  an  organizational  structure  for  the  threat  model  and 
Identifies  the  impact  of  the  battlefield  environment  on  RED  and  BLUE.  The  time 
framework  provides  a  way  to  follow  the  dynamics  of  the  battlefield  and  to  discrim¬ 
inate  events  and  activity  levels  in  the  spatial  framework.  By  spreading  the  threat 
model  information  out  over  a  time  as  well  as  a  spatial  framework,  individual  events 
and  patterns  can  be  discriminated. 

Time  dependencies  can  be  shown  in  the  threat  model  by  labeling  information  with 
date-time  tags  or  by  placing  information  in  a  series  of  overlays  organized  by  time- 
window  boundaries. 
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Thinking  white  is  always  done  In  terms  of  a  geographic  framework.  1  he  boundaries 
for  thinking  wh'to  depend  on  the  specific  mission  objective  and  could  take  the  form 

of: 

•  World  region. 

•  Country. 

•  Local  area. 

•  Township. 

a  Military  district, 
a  Area  of  Interest, 
a  Area  of  influence. 

•  Zone. 

•  Specific  map  coordinates. 

The  geographic  framework  can  be  either  3-dimensional  or  ?-dimensional,  or 
represented  by  a  point  reference  such  as  a  name  or  area  identifier. 

Thinking  white  also  involves  exploiting  the  knowledge  of  the  battlefield  environment. 
Environmental  knowledge  used  in  white  thinking  includes: 

•  Terrain. 

•  Weather. 

a  Population, 
a  Economy. 

a  Energy  and  natural  resources, 
a  Academia  and  technology, 
a  Transportation, 
a  Communications, 
a  Political  and  military  organizations. 

Much  of  this  information  is  available  from  intelligence  and  non- intelligence  sources, 
and  can  be  referenced  to  the  geographic  framework. 

The  focus  of  white  thinking  concerns  the  three  major  areas  of: 

a  Lines  of  communication. 

a  Battlefield  climatic  conditions. 

a  Uncertainty. 

The  following  subsections  discuss  the  specific  white  thinking  techniques  that 
address  these  areas. 
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0.3.2. 1  Linns  of  Communication 

Because  the  Information  volume  associated  with  the  white  element  can  be  massive, 
ways  of  simplifying  the  access  to  this  information  are  needed,  A  principal  technique 
tor  linking  environmental  Information  together  within  the  white  framework  is  the  notion 
of  "lines  of  communication."  For  example: 

•  Population  centers  are  linked  by  roads,  railways,  waterways,  airline  routes,  and 
communications  systems. 

•  Elements  of  an  organization  are  linked  by  lines  of  authority  and  physical  commun¬ 
ication  networks. 

•  Energy  producers  are  linked  to  consumers  through  power  distribution  systems. 

•  Ethnic  groups  are  linked  thn  ugh  culture,  language,  economic,  and  political  lines. 

•  Technology  users  are  linked  with  the  acadenvc  and  research  organizations  that 
produce  that  technology. 

•  Military  units  use  doctrine,  operations,  and  tactics  that  are  passed  along  through 
training  and  command  lines.  Units  are  sustained  through  transportation  lines. 


6.3. 2.2  Battlefield  Climatic  Conditions 

White  thinking  is  used  when  considering  climatic  effects  on  the  battlefield  courses  of 
action.  Climate  is  used  in  a  general  sense  to  refer  to  any  form  of  environmental 
state  that  impacts  on  battlefield  activities.  Battlefield  climatic  conditions  can 
include: 

•  Weather  conditions  (effects  on  personnel,  operations,  equipment,  collection  sys¬ 
tems,  transportation,  supply  expenditures,  etc.). 

•  Terrain  (foliage  for  cover  and  concealment,  communications  effects,  cross¬ 
country  irafficability,  etc.). 

•  Electromagnetic  propagation  conditions  (noise,  interference,  crowding,  blocking, 
etc.). 

«  Social  climate  of  local  population  (reaction  to  enemy  or  friendly  forces,  likelihood 
of  partisan  forces,  sabotage,  etc.). 

•  Economic  climate  (commercial,  industrial  resources  of  area,  economic  stability  of 
local  area). 

c  Political  climate  (political  organization,  stability,  cohesiveness,  goals,  etc.). 

•  Military  climate  (readiness,  professionalism,  ideology,  motivation,  etc.). 

•  International  climate  (alliances,  treaties,  mutual  interests  that  might  involve 
Intervention,  disruption,  or  support  from  other  nations). 

Some  factors,  such  as  International  climate,  are  treated  primarily  at  the  strategic 
level,  but  the  tactical  intelligence  officer  must  understand  their  effects  on  the  bat¬ 
tlefield  conditions.  In  pre-hostility  situations,  these  factors  can  be  more  significant 
because  they  will  bear  on  the  immediacy  of  the  threat  and  possible  escalation  of  the 
danger  In  .he  threat  situation. 
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u.3.2.3  Uncertainty  in  White  thinking 


Uncertainties  in  white  information  can  have  critical  impacts  on  the  outcome  of  the 
battle  or  on  strategic  projections,  locations  ot  strategic  targets,  even  if  accurately 
plotted  can  cause  problems  if  the  map  is  inaccurat  The  time  period  of  interest  can 
Impact  the  accuracy  of  prodictions,  if  the  time  period  of  interest  is  very  extended, 
predictions  of  weather  conditions  are  likely  to  bo  bused  on  general  weather  trends 
or  historical  averages  and  will  therefore  be  inaccurate. 

For  a  large,  generally  defined  battlefield,  analysts  may  have  to  rely  on  general,  pos¬ 
sibly  imprecise  information.  Freq  jently,  only  statistical  information  (such  as  demo¬ 
graphics)  are  available  to  describe  the  population  and  econorale  climate  ot  the  coun¬ 
try  area  It  Is  Important  to  identify  what  environmental  conditions  will  be  significant 
to  specific  missions  so  that  collection  resources  can  oe  tasked  to  provide  greater 
detail  and  accuracy. 


6.3.3  THE  RED  PERSPECTIVE 

The  RED  perspective  of  the  threat  model  must  be  anchored  in  the  real  world  where 
enemy  forces  are  implementing  the  political  and  military  policies  of  the  foreign 
nation.-,  that  generate  those  policies.  The  enemy  combat  force  is  an  extension  of 
political  policies  that  are  the  source  of  the  conflict  that  creates  the  battlefield 
situation. 

The  initial  step  in  introducing  the  red  element  to  the  threat  model  is  to  plot  the  loca¬ 
tions  and  linkages  of  the  political,  military,  and  combat  organizations  within  the  geo¬ 
graphic  framework  of  the  battlefield.  The  highest  echelon  of  enemy  organizations 
plotted  depends  on  the  echelon  of  interest  of  the  friendly  force  commander.  For 
example,  the  strategic-level  threat  mode!  encompasses  ail  aspects  ot  the  civilian 
and  military  force  structure,  whereas  a  division-level  threat  model  might  show  only 
the  locations  of  the  red  forces  up  to  the  army  level. 

The  organization  of  the  red  elements  follows  some  form  of  hierarchical  subordination 
structure  connected  by  lines  of  communication.  The  lines  of  communication  show  the 
relationship  between  units  for  command,  control,  and  coordination.  These  lines  of 
communication  can  be  overlaid  on  the  geographic  structure  and  related  to  physical 
communication  paths  such  as  roads,  transmission  lines,  or  electromagnet  c  transmis¬ 
sion  paths. 

Within  an  actual  combat  environment,  the  locations  ot  combat  units  vary  dynamically 
Correspondingly,  the  positions  of  units  must  be  periodically  updated  within  the  threat 
nodel. 


6.1.4  THINKING  RED 

To  think  RED  is  to  visualize  the  enemy’s  war  plans.  The  red  war  plan  provides  an 
expected  pattern  of  events  that  helps  In  avoiding  surprise  Departures  from  the  red 
war  plan  can  be  i  ’cognized  as  unusual  activities  that  may  signify  a  need  for  addi¬ 
tional  collection,  warnings  to  the  friendly  force,  or  revisions  to  the  threat  model.  The 
rod  war  plans  define; 
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•  Why  the  enemy  will  fight. 

a  Where  the  enemy  will  fight, 
a  When  the  enemy  will  fight. 

•  What  force  structure  the  enemy  will  use. 

•  What  objectives  the  enemy  hopes  to  achieve. 

•  What  time  table  the  enemy  has  for  accomplishing  objectives. 

•  What  possible  options  the  enemy  v  ill  use. 

a  What  courses  of  action  will  be  followed  to  achieve  objectives. 

•  What  operations/tactics  the  enemy  w>i!  use  in  pursuing  courses  of  action. 

•  How  the  onemy  views  his  strengths. 

•  How  the  enemy  views  his  weaknesses. 

•  How  the  enemy  views  the  strengths  and  weaknesses  of  blue  forces. 

A  large  part  of  the  strategic  1  ielligence  products  of  Army  and  other  national  intelli¬ 
gence  organizations  are  concerned  with  answering  these  questions. 

Thinking  from  inside  the  enemy’s  war  plans  is  a  powerful  tool  for  reducing  the  number 
of  potential  battlefield  options  that  are  considered  in  preparing  an  estimate.  Every 
hypothetical  option  can  be  tested  by  asking  how  it  fits  into  the  enemy’s  overall  war 
plans,  and  then  eliminated,  If  the  option  does  not  fit. 

There  are  three  techniques  for  filling  out  the  red  warplan; 

•  Structuring  the  war  plan  using  enemy  doctrine. 

•  Exploiting  intelligence  sources  to  fill  in  details. 

•  Using  analogies  to  fill  in  missing  information. 

Because  the  enemy  is  guided  by  their  doctrine,  strategy,  operational  an,  and  tactics, 
thinking  red  must  begin  with  enemy  doctrine. 


0.3.4. 7  Enemy  Doctrine 

Doctrine  is  a  term  tnat  is  typically  used  to  refer  to  the  fundamental  principles  that 
guide  the  actions  of  a  military  force.  Hov  iver,  from  the  enemy  perspective,  the  term 
doctrine  has  a  specific  meaning  that  has  no  counterpart  in  our  armed  forces.  Enemy 
doctrine  is  an  officially  accepted  system  of  views  concerning  the  nature  of  war, 
methods  for  its  conduct,  and  preparation  of  the  country  and  army  for  war.  At  a 
minimum,  enemy  doctrine  answers  the  questions: 

a  Who  will  be  faced  in  the  war? 

a  What  will  be  the  nature,  goals,  and  missions  in  the  war? 
a  What  armed  forces  will  be  needed  In  such  a  war? 
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•  In  what  direction  should  military  development  be  carded  out? 

•  How  will  preparations  for  war  be  implemented? 

•  What  methods  will  bo  needed  to  wage  war? 

Doctrine  originates  at  the  highest  political  level,  and  the  formulation  of  doctrine  is 
concerned  with  more  than  military  factors.  Enemy  military  doctrine  represents  the 
union  of  politics  and  science  in  support  of  national  objectives.  Enemy  doctrine  is 
concerned  with  the  future  war  and  therefore  precedes  the  development  of  actual 
military  capabilities.  Because  doctrine  may  precede  military  capabilities  by  as  much 
as  several  years,  it  can  be  exploited  as  a  framework  for  the  threat  model  in  a 
peacetime  environment.  Understanding  enemy  doctiine  is  fundamental  to  strategic 
intelligence. 

Military  strategy  is  the  implementation  of  enemy  doctrine.  Military  strategy  is  con¬ 
cerned  with  the  preparation  and  use  of  forces  in  war.  The  enemy  combat  situation  is 
guided  by  military  strategy,  not  doctrine. 

Operational  art  is  concerned  with  preparing  for  and  conducting  joint  and  independent 
operations.  Operational  art  determines  tactical  missions  and  the  role  and  place  of 
units  and  formations  in  achieving  operational  goals. 

Tactics  are  concerned  with  the  fundamentals  of  preparing  for  and  conducting  combat 
operations  by  units  and  formations.  The  meaning  of  tactics,  from  the  enemy  perspec¬ 
tive,  Is  similar  to  our  own. 

The  red  war  plan  evolves  in  a  top-down  manner  from  doctrine.  Without  a  broad,  in- 
depth  knowledge  of  enemy  doctrine,  an  analyst  will  not  have  an  adequate  foundation 
for  developing  the  red  perspective  of  the  threat  model. 


Q.3.4.2  Intelligence  Sources 

iefnlli/nonoo  aruifano  non  fill  iri  mi  ioh  nf  thn  o  r>  r»  m  i  /  umr  rvlon  itomc  ^tratonir  infoMl  — 

mviiniy^iu.u  uwuivvnj  . . . .  J  .  ~  —  --  '-*•  r  •  ~  ■  w  ^  ~  ~  ^  ~  l,  ’ - 

gence  sources  provide  studies  of  enemy  doctrine,  strategy,  weapon  system  charac¬ 
teristics,  and  general  information  on  the  battlefield  environment.  Strategic  analyses 
of  exercises  can  supply  details  on  how  the  enemy  trains  and  operates.  Much  of  the 
procedural  aspects  of  training  will  carry  over  into  battlefield  operations  and  tactics. 


Intelligence  Is  continually  produced  in  peacetime  and  in  wartime  on  enemy  unit  com¬ 
position,  communications,  electronic  warfare  capabilities,  and  characteristics  and 
performance  of  weapon  systems.  By  knowing  the  capabilities  of  combat  systems,  it 
is  possible  to  predict  how  the  enemy  might  plan  to  use  such  a  system  in  the  battle¬ 
field.  This  technique  is  used  to  fill  in  the  threat  model  from  the  "bottom  up." 


6, 3.4,3  Using  Analogies 

Analysts  use  analogies  to  fill  In  missing  information.  For  example,  if  they  have  no 
information  concerning  some  aspect  of  a  red  motorized  unit,  they  may  assume  that  it 
has  some  commonalities  with  a  Blue  mechanized  unit  and  then  draw  an  analogy  from 
this  assumption  to  fill  in  the  missing  Information.  This  technique  is  referred  to  as  mir¬ 
ror  Imaging. 
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Mirror  imaging  can  be  a  powerful  technique  for  filling  in  missing  information  when  no 
other  source  exists.  However,  It  is  always  possible  that  unknown  factors  will  invali¬ 
date  the  mirror-imaging  assumptions.  For  instance,  assumptions  that  Soviet  doctrine 
and  tactics  will  transfer  Intact  to  a  Warsaw  Pact  unit  is  generally  valid  because 
there  Is  an  environmental  similarity  between  the  two  forces.  However,  the  same 
assumption  cannot  be  made  about  the  use  of  those  same  tactics  in  a  Southwest 
Aslan  country  because  the  environment  and  the  culture  of  the  forces  are  so  dif¬ 
ferent. 

A  common  pitfall  of  inexperienced  analysts  is  to  assume  that  the  enemy’s  rationale  is 
a  mirror  image  of  their  own  thinking.  This  kind  of  error  can  lead  to  serious  conse¬ 
quences  for  the  friendly  force  if  decisions  are  based  on  that  rationale.  It  is  for  this 
reason  that  analysts  must  always  Identify  their  assumptions  and  sources  of  infor¬ 
mation  when  developing  the  red  war  plan.  The  need  for  mirror-imaging  is  reduced  in 
proportion  to  the  analyst’s  background  knowledge  concerning  the  ideology,  culture, 
language,  doctrine,  training,  and  history  of  the  enemy.  The  analyst’s  goal  is  to  piace 
him-  or  herself  In  the  role  of  the  enemy  and,  In  fact,  to  think  red. 


6.3.5  THE  BLUE  PERSPECTIVE 

The  threat  model  takes  on  Its  full  meaning  when  the  BLUE  perspective  is  introduced 
along  with  WHITE  and  RED.  The  targets  of  the  red  threat  are  BLUE  elements.  The 
components  of  the  friendly  force  are  also  blue  elements.  The  blue  elements  are 
introduced  into  the  threat  model  by  plotting  the  location  of  strategic  targets  in  the 
white  framework. 


6.3.6  THINKING  BLUE 

Blue  thinking  is  founded  on  the  principles  of  war.  These  principles  form  the  opera¬ 
tional  framework  for  military  actions.  The  principles  of  war  are  not  rules;  rather, 
when  understood  and  applied^  the  principles  shouio  stimulate  thought  and  enhance 
flexibility  of  action.  From  U.S.  Army  Field  Manual  100-1,  the  nine  principles  are: 

•  Objective — Every  military  operation  should  be  directed  toward  a  clearly  defined, 
decisive,  and  attainable  objective. 

•  Offensive-  -Seize,  retain,  and  exploit  the  initiative. 

a  Mass — Concentrate  combat  power  at  the  decisive  place  and  time. 

•  Economy  ot  Force — Allocate  minimum  essential  combat  power  to  secondary 
efforts. 

•  Maneuver--Placa  the  enemy  in  a  position  of  disadvantage  through  the  flexible 
application  of  combat  power. 

•  Unity  of  Command--For  every  objective,  there  should  be  unity  of  effort  under 
one  responsible  commander. 

•  Security — Never  permit  the  enemy  to  acquire  an  unexpected  advantage. 

•  Surprise — Strike  the  enemy  at  a  time  and/or  place  and  in  r.  manner  for  which  he 
is  unprepared. 
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•  Simplicity--Prepare  clear,  uncomplicated  plans  and  clear,  concise  orders  to 
Insure  thorough  understanding. 


The  principles  of  war  define  a  structure  for  blue  thinking.  Each  principle  defines  a 
potential  use  of  threat  model  information. 

Thinking  blue  requires  analysts  to  place  themselves  in  the  role  of  the  intelligence 
user.  This  often  means  a  total  change  of  perspective  from  developing  what  the 
enemy  Is  doing,  and  thinking  instead  in  terms  of  blue  force  actions.  Changing  to  the 
blue  perspective  means  that  analysts  must  think  in  the  same  time  context,  level  of 
detail,  and  terminology  of  the  person  using  the  threat  model  information. 

Thinking  blue  involves  the  following  four  combinations  of  perspectives: 

•  RED  attacks  BLUE. 

Thinking  BLUE  in  the  defense  requires  looking  at  the  RED  war  plan  from  the  per¬ 
spective  of  how  the  friendly  force  is  a  target.  The  information  drawn  from  the 
threat  model  is  from  the  perspective  of  warnings  and  level  of  danger  from  RED 
attack. 

•  BLUE  attacks  RED. 

Thinking  BLUE  in  the  offense  requires  looking  at  the  RED  war  plan  in  verms  of 
weaknesses,  vulnerabilities,  strengths,  and  critical  points  in  courses  of  achon. 

0  WHITE  affects  RED. 

Thinking  of  how  WHITE  affects  RED  requires  looking  at  environmental  conditions 
that  contribute  to  or  detract  from  the  desirability  of  red  options.  These  options 
are  viewed  from  the  BLUE  perspective  of  choosing  the  timing  or  locations  of  con¬ 
flicts. 

0  WHITE  affects  BLUE. 

Thinking  blue  from  the  perspective  of  environmental  conditions  means  looking  at 
the  impact  on  the  effectiveness  of  blue  operations  under  those  conditions. 

Thinking  blue  requires  filling  in  the  threat  model  in  terms  of  different  types  of  infor¬ 
mation,  such  as: 

0  What  options  are  being  considered  by  the  friendly  force? 

•  Is  a  blue  unit  in  danger  of  attack?  Is  a  warning  needed? 

0  Is  there  a  possibility  of  a  new  option  in  the  red  war  plan?  Should  this  option  be 
known  to  avoid  surprise? 

0  Are  there  technological  breakthroughs  that  might  change  existing  forecasts? 

0  Are  there  new  political  or  military  personalities  that  may  altei  the  red  war  plan? 

•  Is  there  a  significant  change  in  the  mass  or  concentration  of  red  forces?  What 
Information  is  needed  by  the  biue  force  to  execute  a  countering  change? 

0  What  does  the  enemy  know  about  the  blue  war  plan?  What  can  be  done  to 
cover  or  deceive  the  enemy  about  this  plan? 
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•  What  are  the  critical  areas  of  uncertainty?  What  collection  assets  can  be 
directed  at  these  areas? 

•  What  information  is  needed  to  carry  out  the  missions  in  the  operational  plan? 

By  rehearsing  the  types  of  questions  that  the  Intelligence  user  might  ask,  it  is  possi¬ 
ble  to  anticipate  the  types  of  information  needed  to  fill  in  the  threat  model. 

Because  the  threat  model  is  dynamic,  it  is  never  completed.  However,  it  is  possible 
to  "freeze"  it  at  any  given  time  period  to  use  for  predictive  purposes.  This  is  done 
by  selecting  the  appropriate  time  window  for  closer  examination.  The  act  of 
developing  the  threat  model  as  well  as  a  particular  time  window,  serves  many  needs. 


6.4  Uses  of  the  Threat  Model 

Threat  modeling  Is  the  act  of  creating  a  mental  picture  of  the  battlefield  in  order  to 
predict  the  outcome  of  combat  missions.  A  threat  model  of  the  battlefield  contains 
more  Information  than  would  be  possible  by  simply  accumulating  facts  from  available 
Information  sources  and  in  this  capacity,  it  serves  many  different  needs: 

6.4. 7  THE  NEED  FOR  PIECING  INFORMATION  TOGETHER 

The  threat  model  is  a  mental  framework  used  to  piece  information  together,  to  aid  in 
remembering,  to  identify  missing  information,  to  speculate  and  predict,  and  to  do 
problem  solving  for  finding  information. 

In  piecing  information  together,  the  threat  model  must  address  the  following  issues  if 
It  is  to  effectively  portray  the  battlefield  knowledge: 

•  Illustrate  current  knowledge  of  the  enemy  forces,  including  their  strength/  capa¬ 
bilities,  disposition,  combat  effectiveness,  and  readiness  state. 

«  Depict  strategy  of  enemy  forces,  including  their  intentions,  commitment  of 
forces,  doctrine  and  ideology,  employment  of  operational  techniques,  equipment 
and  tactics,  and  intermediate  objectives  and  timing  of  events. 

■  Identify  areas  of  uncertainty  and  assumptions  used  to  fill-in  or  compensate  for 
shortfalls. 

«  Provide  logic  for  indicators  that  can  be  used  to  predict,  warn,  and  monitor  the 
execution  of  a  threat  against  friendly  forces. 

•  Qualify  the  credibility  of  the  enemy  force  to  carry  out  a  threat  under  contingen¬ 
cies  of  environmental  factors  (weather,  political,  economic),  enemy’s  level  of 
uncertainty  of  success,  and  friendly  force  options  for  intervention  and  response. 

•  Provide  breadth  of  scope  and  levels  of  detail  necessary  to  see  opportunities  for 
controlling  threat  and  achieving  desirable  outcomes  for  friendly  forces.  The 
types  of  information  produced  by  the  model  should  ensure  both  understanding 
and  detail  ne  essary  for  operation  of  friendly  forces. 
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6.4.2  THE  NEED  FOR  MEASUREMENTS 


The  threat  model  provides  information  that  may  not  be  otherwise  available.  Most 
importantly,  the  threat  model  provides  ways  to  make  measurements  that  quantify  the 
danger  and  risk  present  in  the  threat  situation.  Changes  in  the  model  over  time  are 
used  to  measu>e  trends,  identify  patterns,  and  detect  changes  in  activity  levels. 


6.4.3  THE  NEED  FOR  COMMUNICATING  AND  EXTERNALIZING  INFORMATION 

Because  the  information  contained  in  the  threat  model  has  been  systematically 
represented,  It  is  possible  to  communicate  and  externalize  the  information  contained 
In  the  model.  This  is  an  extremely  important  aspect  of  the  threat  model,  since 
analysts  must  communicate  their  products  to  the  intelligence  user  in  a  timely,  accu¬ 
rate,  and  appropriate  (tailored  for  the  user)  manner. 

Being  able  to  externalize  the  threat  model  information  implies  a  translation  of  the 
analyst’s  mental  image  of  the  battlefield  into  language,  symbols,  and  graphics.  This 
aspect  is  vital  since  the  amount  of  information  represented  by  the  threat  model  is 
well  beyond  the  limits  of  the  human  memory. 


6.4.4  THE  NEED  FOR  DIVIDING  THE  WORKLOAD 

The  threat  model  provides  an  organizational  structure  for  dividing  up  the  analytical 
work  load.  Individual  analysts  can  share  in  the  building  and  maintaining  of  the  infor¬ 
mation  in  the  model,  and  can  also  share  in  the  benefits  of  the  information  provided  by 
the  model. 


6.4.5  THE  NEED  FOR  A  COLLECTIVE  MEMORY 

tkq  threat  mods!  of  rr.odern  warfare  notions  is  so  complex  and  extensive  in  Its  .scope 
and  coverage  of  massive  volumes  of  data  that  many  analysts  are  required  to  create 
and  maintain  the  model.  No  one  analyst  could  hope  to  memorize  the  entire  scope  of 
the  model  information.  The  threat  model  thus  becomes  a  form  of  collective  memory 
between  cooperating  analysts  and  the  operations  personnel  that  the  model  supports. 

The  collective  analytical  team  is  created  by  exploiting  specialized  knowledge  in: 

•  country  area  knowledge  for  creating  environmental  framework 

•  technical  knowledge  for  interpretation  of  indicators  from  signal,  imagery,  and 
human  collection  resources 

•  geographic  partitioning  for  distributing  work  load 

•  technical  specialization  in  interpretation  of  data  with  high  levels  of  uncertainty 

•  technical  specialization  in  modeling  of  physical  systems 

•  operational  exploitation  of  model  information  in  user  areas. 
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6.4.6  THE  NEED  TO  EXTERNALIZE  THE  THREAT  MODEL 


The  threat  model,  because  it  cannot  be  easily  dealt  with  in  internal  memory,  must  be 
encoded  for  communication  and  external  storage.  The  most  commonly  recognized 
forms  are  seen  in: 

•  Threat  estimates. 

•  Situation  displays  over  map  backgrounds. 

•  Order  of  battle  databases. 

These  forms  are  only  partial  threat  models  but  represent  consensus  views  of  an 
analytical  team.  Also  representing  part  of  the  externalized  threat  model  because 
they  represent  community  knowledge  are: 

•  Documented  doctrine  and  tactics  of  opposing  forces. 

•  Documented  characteristics  and  capabilities  of  weapon  systems. 

•  Maps. 

•  Intelligence,  military,  and  state  department  message  traffic. 

•  Previous  intelligence  products. 

•  Intelligence  lore  transmitted  through  oral  tradition. 

•  Technical  knowledge  from  npn-intelligence  sources. 

•  Ethno-graphic  knowledge  available  from  non-intelligence  sources. 

•  Current  events  knowledge  from  open  sources  (newspapers,  periodicals,  publica¬ 
tions,  wire  services,  broadcasts). 

These  secondary  knowledge  sources  are  effective  as  a  component  of  the  threat 
model  in  direct  proportion  to  their  circulation  and  accessibility  to  analysts. 

Because  the  volume  of  externalized  threat  model  data  can  be  enormous,  analysts 
have  intuitively  devised  techniques  for  minimizing  memory  requirements  and  improving 
accessibility,  thereby  achieving  a  form  of  "cognitive  economy."  Examples  of  cogni¬ 
tive  economy  mechanisms  include: 

•  Geographic  plotting  of  time  stamped  messages,  indicators,  and  episodes  for 
filtering,  correlation,  and  activity  level  evaluation. 

•  Environmental  surrogate  models  (such  as  LOC)  for  simplified  environmental  focus. 

•  Predictive  system  templating  with  pre-processed  data  components  to  depict 
composition,  behavior,  characteristics  of  entities. 

•  Predictive  event  templating  to  depict  event  occurrences,  timing,  signature,  and 
relationships  to  indicators. 

•  Symbols  and  iconics  to  represent  standard  entities,  events,  situations  (includes 
military  symbology,  map  topographic  symbology,  unit  names  and  designators, 
etc.). 

Cognitive  economy  is  further  achieved  in  a  more  general  way.  Specifying  that  a 
threat  model  Is  necessary,  u.  iiiai  analysts  have  to  be  trained  in  how  to  develop  and 
use  a  threat  model,  is  not  enough.  To  be  truly  useful,  it  is  necessary  to  establish 


6-15 


criteria  for  evaluating  training  and  analytic  performance.  The  next  chapter 
discusses  issues  for  evaluating  training  and  describes  techniques  for  developing  the 
Ideal  threat  model. 
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7.  EVALUATING  INTELLIGENCE  TRAINING  AND  PERFORMANCE 


7.1  The  Ideal  Analyst 

Tha  characteristics  of  an  intelligence  system  can  be  evaluated  by  comparing  the 
properties  of  an  actual  system  to  those  of  an  ideal  system.  The  requirements  of  an 
Idoal  production  system  may  pose  insurmountable  demands  the  limitations  of 
human  cognitive  processing,  so  that  actual  performance  may  never  approximate  the 
ideal.  However,  a  judicious  extraction  of  concepts  from  an  ideal  production  system, 
In  conji  nction  with  an  understanding  of  the  relevant  cognitive  structures  and 
processes,  can  lead  to  the  identification  of  training  guidelines  and  criteria  for 
evaluating  analytic  performance. 

The  cognitive  model  of  intelligence  analysis  (described  in  Chapters  2  through  5)  was 
derived  from  production  requirements,  data  from  on-the-job  interviews  with  intelli¬ 
gence  analysts,  analysis  of  the  current  cognitive  literature,  and  analysis  of  the  cog¬ 
nitive  perfon  ance  requirements  of  intelligence  analysts.  In  what  follows,  we  exam¬ 
ine  the  components  of  the  ideal  cognitive  performance  model  and  the  ways  in  which 
actual  performance  compares  to  the  ideal. 

To  illustrate  what  the  cognitive  model  can  and  should  accomplish,  an  approach  is  pro¬ 
posed  consisting  of  the  ideal  analyst,  the  ideal  performance,  and  the  ideal  product, 
as  represented  in  Figure  7-1.  In  this  conception,  ideal  states  are  postulated  and 
compared  to  actual  states;  then  the  sources  of  the  differences  between  the  two  are 
Identified.  The  sources  include  human  biases  and  various  types  of  errors  as  well  as 
various  mechanisms  that  are  typically  used  by  humans,  such  as  filtering  and  selection 
of  Information,  stereotyping,  and  assimilation.  These  are  influences  that  have  an 
Impact  on  one  or  all  of  the  constructs  shown  in  Figure  7-1. 

A  goal  is  to  identify  techniques  nr  mechanisms  that  can  compensate  for  sub-optimal 
performance  in  an  actual  operational  environment.  The  steps  necessary  to  identify 
the  Ideal  analyst  performance  are: 

•  Identify  the  task  segments. 

•  Identify  the  cognitive  skills  required  for  each  task  segment. 

•  Make  a  profile  of  the  cognitive  skills  for  each  task  segment. 

•  Relate  the  cognitive  skills  to  the  structures  and  processes  of  the  cognitive 
model. 

•  Identify  the  attributes  associated  with  each  cognitive  structure  and  each  pro¬ 
cess. 

•  Assign  optimum  values  to  the  attributes  identified  above. 

To  identify  compensatory  approaches  for  sub-optimal  performance,  the  following 
steps  are  taken: 

•  Identify  potential  error  sources  that  Impact  on  each  cognitive  structure  and  pro¬ 
cess  and  evaluate  their  significance  within  the  definitional  framework  of  the 
Ideal  performance. 
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•  Select  a  compensation  mechanism  or  technique  that  can  minimize  the  deviations 
between  the  Ideal  and  the  actual  performance  models. 

Task  segments  are  defined  in  terms  of  observable  behavioral  outputs  that  are 
required  to  perform  a  particular  task,  and/or  achieve  a  specific  goal.  Task  segments, 
then,  are  aspects  of  the  production  system  that  are  an  Important  consideration  within 
the  performance  model.  To  understand  the  relationship  between  the  production 
model  and  the  performance  model,  the  cognitive  skills  needed  to  perform  each  of  the 
task  segments  have  to  be  identified,  as  shown  in  Figure  7-2.  For  each  task  segment 
there  are  a  number  of  associated  skills,  although  not  all  skills  are  equally  relevant  or 
Important.  It  is  necessary,  therefore,  to  make  a  profile  of  the  required  cognitive 
skills  for  each  task  segment,  in  which  the  profile  reflects  the  relative  importance  of 
each  skill  in  performing  tha  particular  task  segment.  The  relative  importance  can 
then  be  represented  as  a  value  on  an  arbitrarily  defined  scale,  for  which  the  values 
are  derived  from  the  interview  data  interpreted  through  an  understanding  of  the  cog¬ 
nitive  performance  model. 

Cognitive  skills  are  very  broadly  defined  in  this  context.  They  refer  to  the  use  of 
procedural  and  analytical  skills,  as  well  as  to  the  basic  capacity  of  memory,  the 
capacity  for  attention  and  for  dealing  with  stress.  They  can  also  include  the  effects 
of  physiological,  personality,  and  emotional  variables,  since  all  of  these  have  a 
potential  bearing  on  the  way  task  segments  are  executed. 

Figure  7-2  also  shows  how  each  separate  cognitive  skill  has  an  identified  relation¬ 
ship  to  the  various  structures  and  processes  of  the  cognitive  model.  As  an  example, 
the  cognitive  skills,  structures,  and  processes  involved  in  performing  the  task  seg¬ 
ment  "Hypothesis  Testing",  Involve  the  following  steps: 

•  Selection  of  attributes  of  the  hypothesis. 

•  Translation  of  the  attributes  Into  specific  aspects  of  the  information. 

■  Test  against  the  most  closely  matched  template  within  the  conceptual  model. 

•  Evaluate  the  outcome  of  the  test.  It  is  at  this  point  that  errors  and  biases  may 
affect  performance.  In  the  case  of  hypothesis  testing,  the  most  prominent  bias 
Is  the  tendency  to  seek  confirmatory  evidence,  and  to  ignore  disconfirming  evi¬ 
dence. 

•  If  the  outcome  is  unsatisfactory,  the  steps  above  are  repeated.  If  the  outcome 
Is  satisfactory,  the  task  segment  is  completed. 

•  The  iterative  process  may  continue  over  a  period  of  time,  or  it  may  direct  the 
search  for  new  information  which  could  lead  to  the  generation  of  new 
hypotheses. 

•  In  directing  the  search  for  new  Information,  the  iterative  process  could  be  led 
along  Irrelevant  or  erroneous  paths.  This  is  another  point  in  the  process  where 
the  confirmatory  bias  could  manifest  itself. 

Cognitive  skills  form  the  link  between  the  task  segments  identified  within  the  produc¬ 
tion  system  and  the  structures  and  processes  that  make  up  the  cognitive  model. 
These  skills  can  be  divided  into  categories  that  are  associated  with  the  structures 
and  processes  of  the  cognitive  model,  as  follows: 
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m  Memory  capacity.  1  his  refers  to  nil  memory  stores;  in  the  present  context, 
specifically  long-term  memory  (semantic/f actual  and  episodic)  and  active 
memory.  The  attributes  of  interest  are  size,  adequacy  of  content,  and  complex¬ 
ity  of  the  organization  of  memory. 

•  Storage  and  retrieval.  The  skills  associated  with  storage  and  retrieval  concern 
the  adequacy  of  identifying  relevant  items  to  be  stored  (and,  hence,  remem¬ 
bered)  and  differentiating  them  from  irrelevant  items.  They  also  include  tile  way 
an  item  is  stored  within  the  existing  memory  organization,  a  process  that  is 
nocossary  for  efficient  retrieval.  For  example,  if  an  analyst  stores  information  to 
the  effect  that  a  tank  battalion  has  been  sighted,  but  does  not  stoie  it  within 
the  context  of  location,  time,  and  so  forth,  that  information  is  relatively  useless. 

•  Environment.  The  characteristics  of  the  environment  are  considered  a  part  of  the 
structure  of  the  cognitive  model.  The  relevant  skills,  then,  are  those  that  are 
needed  to  assign  meaning  to  environmental  inputs,  the  most  important  being  pat¬ 
tern  recognition,  tern'  'ate  matching,  and  problem  structuring. 

•  Conceptual  models.  The  skills  associated  with  conceptual  models  are  those 
responsible  tor  the  development  and  use  of  such  models,  ns  discussed  in 
Chapter  3.  They  also  include  communications  skills,  since  shared  conceptual 
models  are  required  for  efficient  communication. 

•  Information  processing.  This  includes  several  processes  within  the  cognitive 
model  that  correspond  to  various  procedural  and  a n a i y t i c a !  skills. 

Each  of  the  above  categories  can  be  represented  as  a  set  of  attributes,  where  indi¬ 
vidual  differences  are  represented  in  terms  of  the  values  of  each  attribute.  For 
example,  the  capacity  of  active  memory  can  be  quite  large  for  some  individuals,  or 
quite  limited  for  others.  Size,  therefore,  is  an  attribute  which  can  take  on  different 
values.  The  value  of  an  attribute  (as  expressed  in  a  particular  individual)  h^s  impli¬ 
cations  for  understanding  performance.  In  the  case  of  active  memory,  the  value  on 
the  size  attribute  will  be  positively  correlated  with  attention  span  (a  performance 
variable).  For  the  ideal  analyst,  optimum  values  on  attributes  must  be  evaluated  for 
the  goals  under  consideration.  Optimum  values  are  not  necessarily  the  same  as  max¬ 
imum  values.  In  some  cases,  a  maximum  value  could  interfere  with  task  performance. 
For  example,  during  hypothesis  generation  (a  task  segment),  it  is  quite  possible  to 
have  so  much  information  in  active  memory  that  too  many  hypotheses  are  generated, 
a  process  that  could  delay  decision  making  and  interfere  with  the  goal  of  reducing 
threat.  It  is  important,  therefore,  to  define  optimum  values  tor  specific  mission  goals 
within  the  context  of  the  triad  that  consists  of  threat  (red  perspective),  environment 
(white  perspective),  and  control  (blue  perspective).  The  cognitive  model  of  the  ideal 
analyst,  then,  is  defined  in  terms  of  sets  of  optimum  values  tor  each  cognitive  strut:  • 
ture  and  process,  where  the  values  are  dynamically  adaptive  to  mission  goals. 


7.2  The  Ideal  Product  as  Threat  Model 

The  "Ideal  product"  is  a  concept  developed  by  I  ogicon  researchers  after  extensive 
investigation  into  the  nature  of  intelligence  production  and  the  cognitive  processes 
of  intelligence  analysts. 
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This  research  developed  notion  that  the  threat  analyses  perlormed  by  intelli¬ 
gence  analysts  have  co  underlying  patterns.  These  patterns  are  evidenced  in 

the  information  flow  i  nt-  n  analyst  and  user  through  reporting  and  other  types  of 
Interact'ons. 

This  research  has  established  a  basis  for  a  generic  production  structure  which  meets 
the  analyst's  goa1  of  minimizing  the  users'  uncertainties  by  providing  information 
that  maximizes  their  ability  to  control  threat. 

It  was  also  found  that  the  models  of  threat  had  to  be  conceptualized  in  the  context 
of  particular  users  and  environmental  contexts  to  be  effective.  hus,  impacting  on 
the  intelligence  products  are  the  multiple  dimensions  of  the  battlefield  in  which  indivi¬ 
dual  products  can  assume  format  and  content  variations  withm  various  user  and 
environmental  contexts. 

Since  the  iMTIA  research  was  per  -r  ,  the  concept  of  the  Ideal  product  has  been 
refined  and  formalized  into  the  concept  of  the  threat  model.  The  ideal  threat  model 
la  the  equivalent  of  the  ideal  product.  The  following  sections  describe  in  detail  how 
to  develop  a  threat  model,  i.e.,  how  to  achieve  the  Ideal  product. 


7.3  General  Steps  for  Developing  the  Threat  Model 

The  tnreat  model  is  a  puzzle  fitted  together  from  thousands  of  pieces  of  information 
that  are  related  In  the  context  of  the  battlefield  situation.  The  geographic  frame¬ 
work  of  the  threat  model  and  location  of  friendly  and  enemy  units  is  the  basic  struc¬ 
ture  used  for  relating  the  pieces.  The  time  context  for  each  snapshot  of  the  threat 
modei  determines  which  pieces  should  be  included  and  which  ones  should  be  left  out. 
Developing  the  threat  model  involves  six  steps: 

•  Increasing  the  level  of  detail. 

v  Representing  areas  of  uncertainty. 

«  Plotting  the  location  of  new  information, 
i  icreasing  the  level  of  accuracy, 
o  Stepping  the  model  forward  or  backward  in  timt  . 

•  Physical  scaling. 

The  following  subsections  describe  these  steps. 


7.3.1  INGRES'S/.' VG  THE  LEV  El  OT  DETAIL 

!nfvi (nation  is  accumulated,  the  level  of  detail  in  the  threat  model  must  be 

eased  The  frame  of  reference  for  the  threat  model  is  the  topographic  map.  Two 
mechanisms  can  be  used  for  adding  more  detail  to  the  map:  (1)  increasing  the 
number  of  overlays  or  (2)  using  smaller  scale  maps  for  greater  resolution  of  detail. 
Map  ovclays  can  provide  more  detail  to  the  threat  model  without  obscuring  topo¬ 
graphic  detail.  New  material  can  be  separated  into  overlays  that  pertain  to  specific 
topics.  Aooinn  mure  overlays  adds  greater  detail,  while  removing  overlays  declutters 
the  threat  model 
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7.3.2  REPRESENTING  AREAS  OF  UNCERTAINTY 


As  the  threat  model  is  being  filled  in,  placeholders  and  flags  may  be  used  to  identify 
missing  information  and  areas  of  uncertainty.  For  instance,  a  symbol  for  a  unit  can  be 
represented  on  the  map  or  adjacent  to  it,  even  if  its  location  is  not  known.  The 
ground  trace  of  a  collection  mission  can  be  plotted  on  the  map  even  though  the  infor¬ 
mation  reports  have  not  yet  been  received  from  that  mission.  Placeholders  and  flags 
serve  a  useful  function  when  they  are  readily  distinguishable  from  known  information. 


7.3.3  PLOTTING  THE  LOCATION  OF  NEW  INFORMATION 

New  data  can  represent  new  information,  errors,  redundancies,  or  contradictions  with 
existing  information.  Until  that  information  is  evaluated  and  assimilated  into  the 
threat  model,  it  is  loosely  tied  to  the  threat  model  by  describing  it  as  an  info,  (nation 
event  that  can  be  plotted  with  respect  to  place  and  time  in  the  white  geographic 
framework. 


7.3.4  CHANGING  THE  LEVEL  OF  ACCURACY 

New  information  can  change  the  accuracy  level  of  the  threat  model.  As  new  informa¬ 
tion  is  brought  Into  the  framework  of  the  thieat  model,  it  is  compared  with  the  exist¬ 
ing  Information.  The  credibility  of  the  new  in'  ormation  must  be  weighed  against  the 
credibility  of  the  old  information  before  an  adjustment  in  accuracy  is  made. 


7.3.5  STEPPING  THE  MODEL  FORWARD  (  R  BACKWARD  IN  TIME 

Because  the  battlefield  is  dynamic,  a  static  threat  model  is  not  adequate.  The  threat 
model  is  made  dynamic  by  the  process  of  creating  snapshots  of  the  threat  model. 
Each  snapshot  of  the  threat  model  represents  some  defined  time  period  A  sequence 
of  successive  snapshots  is  an  animated  view  of  the  battlefield. 

The  value  of  using  the  time  dynamics  of  the  threat  model  is  this:  Features  of  the 
threat  model  that  are  not  dynamic  can  be  transferred  from  one  snapshot  to  the  next 
without  effort  (for  example,  the  performance  characteristics  of  a  specific  weapon 
system  will  not  change  from  one  hour  to  the  r  ;xt,  although  its  physical  location  might 
change)  and  the  dynamic  features  can  aisc  be  updated  without  effort  or  represented 
as  predictions. 


Stepping  the  model  backward  in  time  allows  many  pieces  of  information  to  be  added 
that  wouid  otherwise  not  be  included  in  the  ttireat  model.  For  example,  mformat  on 
from  prior  battles,  exercises,  or  training  doctr.ne  are  useful  as  a  historical  base  for 
developing  the  model.  The  actual  threat  model  might  incorporate  information  about 
weapons  and  tactics  that  dates  back  to  information  sources  that  are  years  old.  In 
effect,  the  principle  of  "Inheritance"  is  used  tc  fill  in  information  from  the  historical 
base. 

An  important  value  of  ii:e  threat  model  is  thot  the  snapshots  can  be  extended  into 
the  future  as  an  aid  to  envisioning  and  predicting  the  course  of  the  battle  and  the 
expected  outcomes  of  missions.  Filling  in  the  future  time  snapshots  requires 
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considerations  of  plans,  trends,  contingencies,  probabilities,  and  ::ts 


7.3.6  PHYSICAL  SCALING 

Events  and  behaviors  are  frequently  difficult  to  observe  at  actual  scale.  The  threat 
model  can  be  scaled  to  obse.ve  such  behaviors  in  terms  of  both  space  or  time. 
Space  Is  scaled  in  terms  of  a  topographic  map.  lime  can  be  scaled  to  any  period  of 
Interest  or  to  encompass  dynamic  behaviors. 

For  example,  with  a  scaled  representation,  it  is  possible  to  determine  how  far  battle¬ 
field  entities  can  see,  shoot,  move,  communicate,  or  jam 

Physical  scaling  makes  it  possible  to  detect 

•  Trends. 

•  Patterns. 

•  Missing  information. 

•  Errors. 

Map-reading  skills  are  a  critical  requirement  for  accurate  physical  scaling  in  threat 
modeling.  These  skills  are  needed  to  determine  line-of-sight  distances,  fields  of  fire, 
and  various  aspects  of  mooility. 


7.4  Dimensions  of  the  Threat  Model 

The  threat  model  can  be  systematically  oartitioned  so  the.  the  massive  volume  o f 
Information  represented  by  the  model  can  be  managed.  Much  of  this  information  can 
be  translated  .nto  text,  symbols,  and  graphics,  but  much  of  it  remains  in  the  analyst's 
head  Without  additional  structuring  beyond  white,  red,  and  blue  thinking  (see 
Chapter  6),  the  analyst  wouia  stiii  have  great  difficulty  in  coping  with  the  massive 
volume  of  information. 

Techniques  are  available  tc  partition  the  battlefield  information  so  that  it  is  easier  lo 
remember  and  manage.  These  partitioning  techniques  also  provide  a  systematic 
structure  that  can  be  used  when  dividing  the  analytical  workload  in  an  analytical 
team.  This  organizational  structure  can  also  be  implemented  in  automated  data 
bases  that  support  threat  mede'ing. 

The  partltionmg  techniques  include: 

•  Creating  descriptions  at  increasing  levels  of  detail. 

•  Allowing  descriptions  with  varying  levels  of  precision 

•  Partitioning  by  geography. 

•  Class’fying  areas  by  topographic  features. 

•  Indexing  objects  by  battlefield  depth. 

e  Organizing  information  by  time  windows. 
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•  Organizing  information  by  relevance  to  echelon  level. 

•  Integrating  events  and  information  into  aggregate  elements. 

The  following  subsections  describe  these  techniques  for  dividing  and  conquering  the 
complexity  of  the  threat  model. 


7  4. 1  INCREASING  LEVELS  OF  DETAIL 

Not  ail  intelligence  users  need  the  same  level  of  detail.  The  threat  model  can  be 
stratified  into  overlapping  leveis  of  detail.  For  instance,  the  opposing  force  could  be 
described  in  terms  of  its  hierarchical  structure,  with  increasing  levels  of  detail 
corresponding  to  lower  and  lower  levels  of  the  hierarchy.  Each  level  mignt 
correspond  to  a  different  type  of  blue  usage. 


7.4.2  VARYING  THE  LEVEL  OF  PRECISION 

Not  all  aA  the  information  in  the  threat  model  needs  to  be  at  the  same  level  of  preci¬ 
sion.  For  instance,  if  only  two  of  the  three  battalions  In  an  enemy  regiment  have 
been  detected,  the  third  can  be  assumed  to  be  present  because  of  enemy  doctrine. 
The  third  unit  can  be  represented  In  the  threat  model,  but  at  a  lower  level  of  preci¬ 
sion  in  order  to  make  the  model  as  complete  as  possible. 

Completeness,  as  defined  by  the  mission  objectives,  is  more  important  in  threat 
modeling  than  consistency  in  precision.  Force  strengths  cannot  be  estimated  using 
the  threat  modei  unless  the  model  is  complete.  Analysts  must  know  and  report  the 
precision  of  the  representations  in  the  threat  model. 

The  most  precise  Information  is  not  necessary  to  begin  formulating  the  threat  model. 
The  threat  model  can  be  initially  formed  by  doctrine  and  then  refined  as  more  precise 
information  is  obtained.  Understanding  the  needed  precision  levels  will  help  analysts 
focus  the  collection  and  analysis  activities  to  improve  the  threat  model. 


7.4.3  PARTITIONING  BY  GEOGRAPHY 

A  straightforward  means  of  dividing  the  information  level  of  the  threat  model  is  to 
divide  it  by  geographic  sectors.  This  technique  is  used  as  a  matter  of  course  in  the 
assignment  of  areas  of  influence  and  areas  of  interest  by  echelon. 


7.4.4  CLASSIFYING  BY  TOPOGRAPHIC  FEATURES 

Information  about  topographic  features  of  the  battlefield  can  be  classified  so  that  it 
applies  directly  to  tho  type  of  combat  options  planned  for  the  geographic  an.  a.  This 
type  of  classification  makes  it  easy  to  bring  the  general  background  knowledge  into 
the  threat  model.  Topographic  clossifications  include: 

•  Weather  options  (clear,  inclement,  monsoon,  etc.). 

•  Day/night  options. 
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•  Open  terrain  options. 

•  Forest. 

•  Arctic. 

•  Desert. 

•  Mountain. 

•  Urban. 


7.4.5  INDEXING  BY  BATTLEFIELD  DEPTH 

From  the  blue  perspective  of  the  battlefield,  the  red  area  can  be  viewed  in  succes¬ 
sive  levels  of  depth.  The  depth  index  determines: 

•  What  types  of  objects  are  seen. 

•  How  soon  the  information  on  the  object  Is  needed. 

•  What  level  of  detail  is  needed  on  the  object. 

•  What  source  is  available  to  provide  that  information. 

•  The  level  of  uncertainty  concerning  the  object. 


7.4.6  ORGANIZING  BY  TIME 

The  threat  model  can  be  organized  into  successive  time  snapshots  to  depict  the 
dynamic  changes  of  the  battlefield.  By  limiting  the  extent  of  each  snaoshot,  the 
amount  of  Information  to  be  dealt  with  can  be  controlled. 

Moving  from  one  snapshot  to  the  next  also  purges  Information  whose  value  has  dimin¬ 
ished.  Old  information  can  have  an  inertial  effect  that  delays  the  detection  of 
dynamic  changes  in  the  battlefield. 

Snapshots  of  wider  time  scope  can  be  overlaid  on  narrower  snapshots  in  order  to 
deal  with  information  in  greater  generality  or  with  patterns  that  have  a  slower  rate  of 
change.  The  strategic  snapshot  of  the  battlefield  is  much  wider  than  the  tactical 
snapshot. 


7.4.7  ORGANIZING  BY  RELEVANCE  TO  ECHELON 

The  potential  complexity  of  the  threat  model  Is  reduced  long  before  it  gets  to  the 
tactical  intelligence  analyst.  The  strategic  intelligence  products  that  precede  the 
tactical  intelligence  mission  will  have  defined: 

•  Where  the  battlefield  is. 

•  Who  the  enemy  is. 

•  What  the  enemy’s  strategic  options  are. 
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•  What  the  enemy’s  general  force  structure  Is. 

•  What  the  enemy’s  weapon  technology  is. 

•  What  the  enemy’s  Initial  tactical  resources  are. 

•  What  the  enemy’s  sustaining  capabilities  are. 

These  general  strategic  options  can  be  furthered  stratified  for  the  tactical  threat 
model.  For  instance,  the  threat  model  can  be  stratified  into  the  following  potential 
options: 

•  Conventional  warfare. 

•  Chemical/biological  environment. 

•  Limited  nuclear  environment. 

•  Strategic  nuclear  environment. 


7.4.8  INTEGRATING  THE  ELEMENTS  OF  THE  THREAT  MODEL 

One  major  task  in  developing  an  ideal  product  is  to  integrate  the  elements  of  the 
threat  model  in  a  way  that  will  satisfy  each  user’s  requirements.  The  pieces  of  the 
threat  model  must  be  integrated  into  a  meaningful  representation  of  the  battlef.eld 
that  Is  complete  within  the  mission  context.  Bringing  the  pieces  of  the  threat  model 
puzzle  together  Is  the  process  of  seeing  the  battlefield  in  order  to  accomplish  a  blue 
objective  or  mission. 

Dividing  the  battlefield  into  its  different  dimensions  creates  information  chunks  that 
are  easier  to  remember  and  that  facilitate  threat  modeling.  Time  and  geography  are 
always  the  reference  frameworks  that  relate  the  information  chunks  to  each  other. 


7.4.8. 7  Integrating  for  the  Product 

To  create  an  Intelligence  product,  various  elements  of  the  threat  model  are  selected 
for  integration.  A  time  snapshot  is  selected  that  represents  the  timeframe  within 
which  the  product  user  applies  the  information  to  combat  decis’ons  and  the  geo¬ 
graphic  scope  of  the  information  is  matched  to  the  product  user’s  area  of  influence 
and  area  of  interest. 

Product  context  also  determines  the  level  of  detail  necessary  for  viewing  the  threat 
model  information.  Analysts  must  consider  what  terminology  and  level  of  detail  will  be 
required  for  the  product  user  to  understand  the  information.  For  instance,  the  threat 
model  may  include  information  on  each  known  artillery  position.  However,  the  product 
user  may  only  be  interested  in  the  battalions  represented  by  those  units. 


7. 4.8.2  Integrating  tor  Completeness 

Completeness  within  the  mission  context  Is  the  most  important  aspect  of  threat 
model  integration.  Completeness  means: 
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•  Showing  the  full  strt  igth  of  the  enemy  force  even  though  only  half  of  the  units 
in  the  force  may  have  been  Identified  and  located. 

•  Showing  a  probable  position  of  a  unit  even  thoup!<  there  is  uncertainty  as  to  that 
location. 

•  Showing  the  expected  enemy  course  of  action,  even  if  it  is  a  guess. 


Completeness  Is  what  makes  the  threat  model  most  useful  as  a  decision  aid.  Seeing 
the  battlefield  is  seeing  the  complete  picture  of  the  battlefield  situation,  even  if 
pieces  are  highly  uncertain. 

Certain  kinds  of  tools  have  been  devised  for  keeping  the  threat  model  as  complete 
as  possible.  These  aids  include: 

•  Templates.  A  template  is  a  generalized  model  of  the  composition  of  a  class  of 
unit,  object,  activity,  or  event  on  the  battlefield.  A  template  describes  the  gen¬ 
eral  features  ana  relationships  of  features  to  the  battlefield  environment.  For 
example,  a  template  of  a  type  of  enemy  artillery  battery  would  describe  the 
numbers  of  artillery  pieces,  supporting  equipment,  and  the  general  positioning  of 
the  pieces  when  deployed. 

Many  different  forms  of  templates  are  described  in  the  IPB  process,  including 
doctrinal,  situation,  and  event  templates.  Additional  templates  are  useful  for 
describing  the  signatures  of  various  battlefield  objects  and  activities.  Signature 
templates,  in  particular,  are  useful  in  piecing  together  Information  from  multiple 
sources. 

•  Tracking.  A  track  can  be  used  to  integrate  Information  that  falls  into  multiple 
time  snapshots,  as  in  the  position  of  a  maneuver  unit.  Positions  of  enemy 
maneuver  units  can  be  integrated  by  correlating  individual  position  reports  with 
an  assumed  track  or  route.  The  track  of  a  maneuver  unit  would  appear  as  a  line 
connecting  the  points  of  its  past  iocations. 

Tracking  can  also  apply  to  maintaining  continuity  on  identifying  and  locating  criti¬ 
cal  elements  of  the  enemy  force.  Knowing  that  continuous  information  is  avail¬ 
able  on  enemy  units  Is  a  means  of  reducing  the  risk  of  surprise  or  deception. 

Tracking  can  also  apply  to  integration  of  information  about  stationary  objects 
that  have  dynamic  features.  Tracking  an  airfield  would  involve  following  the 
changes  in  numbers  and  types  of  aircraft,  modifications  (to  runway,  defense,  and 
facilities)  and  activity  types  and  levels. 

Order  of  Battle  data  bases  are  an  example  of  tracking.  These  data  bases  con¬ 
tain  complete  records  of  the  enemy’s  organization,  strength,  composition,  and 
characteristics.  The  form  and  behavior  of  enemy  units  or  organizations  are 
included  in  OB  data  bases. 

Tracking  allows  the  threat  model  to  provide  information  on  patterns  and  trends  of 
battlefield  activities.  Thus,  tracking  can  provide  a  useful  method  for  identifying 
unusual  or  unexpected  enemy  actions  or  events. 

•  Time  Lines.  Courses  of  action  can  be  more  completely  modeled  in  terms  of  time 
duration  and  occurrence  of  key  events  If  the  course  of  action  can  be 
represented  as  a  program  of  activities  and  events  over  time.  For  example,  the 
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event  analysis  matrix,  described  in  the  IPB  proce.  ,,  correlates  expected  events 
and  activities  with  each  NAI  and  TAI  and  adds  the  dimension  of  time. 

•  Networks.  Networks  are  groups  of  units  linked  together  to  serve  some  specific 
function,  such  as  communications,  supply,  command,  or  sharing  of  a  common 
resource.  A  network  may  also  be  used  to  show  the  aggregation  of  subordinate 
units  into  a  higher-level  unit  (as  in  force  composition  or  tables  of  organization 
and  equipment).  Thinking  of  units  as  embedded  in  a  network  can  aid  in  complet¬ 
ing  the  information  in  the  threat  model  even  if  some  of  the  units  in  the  network 
cannot  be  idc  tified  or  located. 

Even  If  full  information  is  not  available  on  the  events  In  the  course  of  action,  the 
program  structure  helps  to  anticipate  the  duration  of  activities  and  the  time¬ 
spacing  between  events.  For  instance,  second-  echelon  staging  as  a  program  of 
activities  may  be  represented  so  as  to  determine  the  time  and  geographic  rela¬ 
tionships  between  indicators  o *  that  activity.  The  same  would  be  true  of  a 
river-crossing  activity.  The  events  leading  up  to  the  river  crossing,  the  schedul¬ 
ing  of  units  crossing  the  river,  and  actions  following  the  crossing  could  be 
treated  as  a  scheduled  program. 


7.5  Making  Measurements  with  the  Threat  Model 

Perhaps  one  of  the  most  useful  functions  of  the  threat  model  is  in  its  capacity  for 
prediction.  Prediction  must  be  preceded  by  accurate  measurements  of  the  various 
elements  of  the  threat  model.  When  integrated  within  the  threat  model,  these  ele¬ 
ments  can  be  measured  In  ways  that  individual  pieces  of  information  cannot. 

Scaled  measurements  can  be  used  to  provide  quantitative  data  on  enemy  capabili¬ 
ties.  Time  measurements  can  be  used  for  predicting  events.  Statistical  measure¬ 
ments  made  on  the  threat  model  can  Identify  unusual  activities,  trends,  and  patterns. 

Three  categories  of  measurement  can  be  addressed  using  the  threat  model.  These 
categories  include: 

•  Force  strength. 

•  Uncertainty. 

•  Risk. 

The  following  subsections  discuss  these  categories. 


7.5.  7  MEASURING  FORCE  STRENGTH 

The  threat  model  can  be  used  for  aggregating  the  strength  of  enemy  forces  now  and 
In  the  future.  The  threat  modei,  because  of  its  geographic  framework,  can  be  used 
to  compare  red  force  strength  to  blue  force  strength  on  an  area  basis.  Force  con¬ 
centration  and  force  ratios  are  key  indicators  used  by  the  commander  to  evaluate 
enemy  Intentions  and  risk. 

The  relative  combat  effectiveness  of  the  enemy  can  be  measured  with  the  aid  of  the 
threat  model.  Doctrine  provides  a  "design  strength"  of  manning  level,  equipment, 
reserves  against  which  actual  fluctuations  can  be  measured.  Making  this  type  of 
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measurement  requires  continual  tracking  of  the  attrition  and  resupply  activities. 


7.5.2  MEASURING  UNCERTAINTY 

Measuring  uncertainty  involves  a  number  of  quantifications.  For  example: 

•  Information  source  credibility. 

•  False  alarm  error  rate. 

•  Miss  error  rate. 

•  Imprecision. 

•  Unexpectedness. 

•  Missing  information. 

•  Inconclusive  information. 

The  ability  to  recognize  these  types  of  uncertainty  in  the  threat  model  is  a  required 
skill  of  the  intelligence  analyst.  The  risks  associated  with  the  uncertainties  due  to 
the  analyst’s  own  skill  limitations  must  also  be  taken  into  account. 

The  following  subsections  describe  briefly  these  areas  of  uncertainty. 


7. 5.2.1  Information  Source  Credibility 

Most  information  in  the  threat  model  will  have  a  validity  established  by  belief  in  the 
truth  of  the  source  and  knowledge  of  conditions  under  which  the  information  would  be 
true.  Information  source  credibility  may  be  based  on  the  historical  success  of  a 
source  or  on  the  engineering  constraints  of  the  system.  The  predictability  of  source 
credibility  is  generally  ba-.cd  on  a  projection  of  the  historical  success  rate  qualified 
by  environmental  conditions. 

There  are  two  rating  scales  associated  with  an  item  of  information,  one  rating  relia¬ 
bility  of  the  information  source,  the  other  rating  the  accuracy  of  the  information. 
These  evaluations  are  discussed  In  detail  in  Field  Manual  30-5,  Combat  Intelligence. 


7 .5.2.2  False  Alarm  Error  Rate 

A  false  alarm  occurs  when  a  data  item  is  reported  erroneously,  for  example,  if  a  sen¬ 
sor  gives  a  false  reading  or  if  data  are  misinterpreted. 

The  false  alarm  error  rate  is  the  ratio  of  false  alarms  to  correct  evaluat.ons.  This 
rate  can  be  heavily  biased  by  the  analyst’s  expectations.  If  events  are  expected, 
they  tend  to  be  perceived  even  if  they  have  not  occurred.  This  type  of  bias  m^es 
analysts  susceptible  to  enemy  deception.  The  ability  to  recognize  situations  where 
false  alarms  are  highly  probaoie  Is  a  skill  that  can  protect  analysts  from  the  effects 
of  enemy  deception. 
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7.5. 2.3  Miss  Error  Rate 


A  miss  error  Is  when  an  event  actually  occurs,  but  Is  not  detected.  Miss  errors  can 
occur  because  of  Incomplete  sensor  coverage,  enemy  cover  and  concealment 
actions,  or  because  of  noise  and  errors  In  Information.  The  miss  error  rate  is  the  ratio 
of  misses  to  correct  evaluations.  The  result  of  having  a  high  miss  error  rate  Is  the 
risk  of  surprise. 

The  ability  to  recognize  situations  with  high  miss  error  rates  Is  a  skill  that  allows  the 
analyst  to  compensate  for  errors  by  diversifying  collection  activities  and  warning 
friendly  forces  to  be  vigilant  for  surprise  enemy  actions. 


7.5.2.4  Imprecision 

Imprecision  is  the  degree  of  uncertainty  about  a  numerical  measurement  or  an  object 
Identification.  Sensor  Imprecision  may  result  In  uncertainties  about  the  location  or 
classification  of  an  enemy  unit.  The  number  of  digits  used  in  listing  the  UTM  coordi¬ 
nates  on  a  location  implicitly  defines  the  level  of  precision  on  the  location. 

Numerical  imprecision  may  also  occur  in  identifying  the  size  of  a  force,  the  perfor¬ 
mance  of  a  weapon  system,  or  the  time  resolution  of  when  an  event  occurred. 

An  imprecise  Identification  involves  ambiguity  in  class'  ration  of  size  or  type  For 
example,  the  terms  "heavy  tank"  and  "battalion-size  jnit"  reflect  imprecision  in 
class  Identification.  Imprecision  may  also  apply  to  ambiguity  of  specific  unit  identi¬ 
ties.  For  example  "a  battalion  of  the  third  armor  division"  is  a  less  precise  identifica¬ 
tion  than  "the  42nd  battalion  of  the  third  armor  division." 

Using  precise  terminology  when  it  is  not  appropriate  may  be  misleading  and  have 
negative  consequences  for  the  friendly  force.  For  example,  transmitting  the  precise 
location  of  a  mobile  enemy  unit  without  a  time  qualifier  may  result  in  the  execution  of 
a  useless  artillery  mission.  Knowing  how  and  when  to  use  precision  is  a  skill  that  the 
intelligence  analyst  needs. 


7.5. 2.5  Unexpectedness 

Unexpectedness  is  the  level  of  departure  of  a  measurement  from  an  expected  value. 
The  expected  value  may  be  based  on  historical  values  or  trends,  doctrine  state¬ 
ments,  or  analogies  if  the  event  has  not  occurred  before. 


7.5. 2.6  Missing  ,nformation 

Missing  information  may  be  gaps  in  battlefield  surveillance,  apparent  missing  features 
in  physical  models  of  battlefield  objects,  missing  inf  mmation  on  physical  attributes  of 
known  features,  or  missing  elements  that  compose  units.  Being  able  to  identify  infor¬ 
mation  as  missing  rather  than  as  imprecise  requires  having  a  template,  mirror  image, 
network,  or  program  structure  to  compare  with  existing  information. 
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7. 5. 2. 7  Inconclusive  Information 

Inconclusive  Information  is  unexpected  and  cannot  be  interpreted  in  terms  of  the 
existing  threat  model  structure.  If  the  unexpected  information  cannot  be  attributed 
to  noise,  normal  environmental  variations,  errors,  or  identified  battlefield  objects,  it 
can  still  be  identified  as  an  information  event  and  retained  for  future  Interpretation. 

An  unidentified  moving  target  detected  by  a  radar  can  be  considered  Inconclusive 
Information  if  it  cannot  be  associated  with  a  particular  enemy  activity.  Inconclusive 
information  can  be  useful  if  it  can  be  used  as  a  measure  of  unusual  activity  or  as  a 
guide  for  directing  future  collection  missions. 

For  example,  construction  activities  in  the  enomy’s  rear  area  may  be  observed,  but 
an  identification  of  the  type  of  construction  may  not  be  possible.  The  event  is  signi¬ 
ficant  if  It  eventually  is  identified  as  a  fortification  for  a  command  post  or  a  missile 
installation. 

The  ability  to  recognize  potentially  significant  information  is  a  skill  that  allows 
analysts  to  piece  together  bits  of  information  that  by  themselves  would  be  meaning¬ 
less.  The  threat  model  provides  the  time  and  space  framework  for  retaining  these 
pieces  of  information  until  they  can  be  Interpreted  in  terms  of  some  aspect  of  the 
enemy  war  plan. 

7.5.3  MEASURING  RISK 

Measuring  risk  involves  merging  risk  factors  with  uncertainty  measurements.  Risk 
factors  include: 

o  Enemy  force  lethality. 

•  Warning  time. 

•  Number  of  enemy  options. 
m  Member  of  friendly  options. 

•  Knowledge  of  environmental  impact  on  red  and  blue. 

Risk  must  be  qualified  by  the  uncertainty  introduced  by  source  credibility,  error 
rates,  and  imprecision  which  affect  risk  factors. 


7.5.3. 1  Measuring  Lethality 

Lethality  is  a  measure  of  the  enemy's  military  cape.bi'ities.  Lethality  might  reflect 
force  size,  weapon  technology,  weapon  performance,  combat  effectiveness,  force 
concentration,  and  sustainability  rate.  The  measurement  of  lethality  is  normally  done 
In  comparison  to  tho  countering  friendly  force.  Changes  in  lethality  are  important 
because  they  can  be  indicators  of  enemy  intentions,  strengths,  or  weaknesses. 

A  commander  must  be  as  certain  as  possible  about  the  lethality  of  the  enemy  force. 
When  there  Is  a  range  of  uncertainty  about  enemy  lethality,  the  commander  must  plan 
for  worst-case  and  best-case  situations  as  well  as  for  nominal  case.  It  is  the 
analyst's  responsibility  to  narrow  the  range  of  uncertainty  about  enemy  lethality  so 
that  the  commander  does  not  have  to  waste  planning  and  mission  resources  on 
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unrealistic  options. 


7.5.3. 2  Measuring  the  Number  of  Enemy  Options 

Although  the  number  of  enemy  options  can  be  potentially  infinite,  the  more  probable 
options  are  the  ones  that  fall  into  doctrinal  patterns  and  that  resemble  patterns 
observed  In  training,  exercises,  or  previous  conflicts.  Plausible  enemy  options  can 
be  tanked  in  order  of  their  closeness  to  doctrinal  patterns.  Other  factors  to  consider 
In  evcluating  enemy  options  Include: 

•  Indications  of  preparations  being  made  tor  multiple  options. 

•  The  mobility  of  the  enemy  force. 

•  The  friendly  force  vulnerabilities  that  the  enemy  is  most  likely  to  try  to  exploit. 


7.5.3.3  Measuring  Warning  Time 

Warning  time  is  the  time  available  to  the  friendly  force  to  prepare  a  response  to  an 
Identified  enemy  threat  The  measure  of  warning  time  must  be  qualified  by  uncer¬ 
tainties  caused  by  false  alarms,  miss  error  rate,  source  credibility,  and  missing  infor¬ 
mation. 

When  exact  scheduling  of  enemy  actions  is  not  known,  the  analyst  can  measure  fac¬ 
tors  that  are  Indicators  of  the  available  warning  time: 

•  Closeness  of  enemy  troops  to  FLOT. 

•  Increased  mobilization  of  reserves. 

•  Increased  activity  in  supply  lines. 

•  Proximity  of  forces  to  readiness  (time  needed  to  complete  final  steps  of 
preparations). 

•  Maneuver  capabilities. 


7 . 5.3.4  Measuring  Friendly  Options 

The  commander  selects  the  combat  options  of  the  friendly  force  following  the  princi¬ 
ples  of  war  and  embodies  these  options  into  the  operational  plan.  The  analyst’s  role 
is  to  make  sure  that  selected  options  are  workable  in  light  of  enemy  actions, 
weather,  and  terrain.  The  threat  model  can  be  used  to  identify  factors  that  make 
selected  opti  ns  more  desirable,  such  as: 

•  Favorable  characteristics  of  friendly  force  mobility  corridors. 

•  Collection  capabilities  to  support  targeting  options. 

•  Enemy  vulnerabilities. 

•  Critical  points  in  enemy  plans. 

•  Operations  security  to  deny  knowledge  of  friendly  plans  to  enemy. 
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7.5.3. 5  Measuring  Knowledge  of  the  Environment. 

Environmental  factors,  such  as  weather  and  terrain,  are  qualifiers  on  the  effective¬ 
ness  of  both  enemy  and  friendly  courses  of  actio  .  Risk  can  be  attributed  to  the 
Impact  of  these  factors  on  the  expected  success  of  a  mission.  Risk  can  also  be 
attributed  to  the  balance  of  environmental  knowledge  between  enemy  and  friendly 
forces  and  their  ability  to  exploit  that  knowledge. 

Uncertainty  about  the  impact  of  environmental  factors  on  friendly  end  enemy  options 
could  cause  the  commander  to  select  non-optimum  courses  of  action.  Analysts  must 
determine  the  impact  of  em  ironmental  factors  on  both  friendly  and  enemy  courses  of 
action  and  convey  that  information  to  the  user  or  commander. 


7.6  Summary 

The  threut  model  is  a  way  of  seeing  the  battlefield.  Analysts  must  use  three  per¬ 
spectives  in  creating  the  threat  model.  In  THINKING  WHITE,  the  battlefield  is  viewed 
in  a  time  and  space  perspective.  By  thinking  white,  analysts  give  the  commander  an 
advantage  in  exploiting  terrain,  weather,  or  resources  of  the  local  region. 

In  THINKING  RED,  the  battlefield  is  seen  from  the  enemy’s  planning  perspective. 
Thinking  red  allows  the  analyst  to  predict  what  the  enemy  will  do  in  given  situations. 

In  THINKING  BLUE,  the  focus  is  on  the  information  required  to  accomplish  the 
commander’s  mission.  Thinking  blue  allows  the  analyst  to  sort  threat  information 
appropriately  —  exploiting  information  for  immediate  needs,  storing  useful  information 
for  later  use,  and  ignoring  information  that  cannot  be  exploited. 

Tables  7-1,  7-2,  and  7-3  represent  checklists  that  summarize  ways  of  thinking  from 
each  of  the  three  perspectives. 
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What  is  the  geography  of  the  battlefield'7 
What  is  the  culture  of  country  area? 

What  is  the  history  of  conflicts  in  this  region? 

What  is  the  attitude  of  the  local  population  toward  the  enemy  force  7 
toward  the  friendly  force? 

What  is  the  academic,  economic,  and  technology  level  of  the  local  population? 
How  Is  the  local  population  armed? 

How  Is  the  population  distributed  in  the  region? 

What  commercial,  Industrial,  and  natural  resources  exist  in  the  region? 

What  loc^1  cultural  features  (e.g.,  airfields,  railways)  can  be  exploited 
for  military  purposes? 

How  predictable  is  the  weather  in  the  area? 

How  does  tire  weather  affect  operations,  personnel,  trafficability,  equipment, 
and  communications? 

During  what  time  frame  will  hostilities  occur? 

How  much  time  Is  available  for  preparation? 

Table  7-1.  Checklist  of  WHITE  Questions 
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What  is  the  enemy’s  doct'lne? 


How  doec  the  enemy  prepare  war  plans? 
What  are  the  enemy’s  objectives? 

What  is  the  enemy’s  strategy? 

What  are  the  enemy’s  operational  plans? 
What  are  the  enemy’s  courses  of  actions? 
What  are  the  enemy’s  tactics? 

How  does  the  enemy  train? 

How  is  the  enemy  equipped? 

How  is  the  enemy  force  sustained? 

How  is  the  enemy  force  structured? 

Table  7-2.  Checklist  of  RED  Questions 


What  are  the  BLUE  principles  of  war? 

What  are  the  friendly  objectives? 

What  are  the  friendly  force  resources? 

What  is  the  commander’s  mission? 

What  resources  does  the  commander  have  at  his  disposal? 
What  are  the  priority  information  needs  of  the  commander? 
What  are  the  Information  needs  to  execute  the  mission? 


Table  7-3.  Checklist  of  BLUE  Questions 


8.  AUTOMATED  SYSTEMS 


The  objective  of  this  chapter  Is  to  describe  how  automated  systems  fit  Into  the  cog¬ 
nitive  processes  of  the  Intelligence  analyst.  There  are  a  limited  number  of  prototype 
automated  systems  that  support  tactical  intelligence  production  and  thero  are 
several  operational  systems  that  currently  support  strategic  intelligence  production. 
Many  more  automated  intelligence  support  systems  are  In  a  requirements  definition  or 


development  stage.  The  developers  of  these  systems  need  an  awareness  of  the 
cognitive  processes  of  the  analysts  who  will  use  their  systems. 

Over  the  course  of  the  IMTIA  study,  site  visits  were  conducted  to  observe  and 
evaluate  the  use  of  existing  automated  systems  that  support  intelligence  production. 
Table  8-1  lists  the  systems  that  were  reviewed  during  the  course  of  this  study. 


SYSTEM 

TYPE 

BETA 

TCAC 

ITEP 

DITB 

AMH 

Tactical,  Testbed 

Tactical,  Development 

♦TENCAP 

♦TENCAP 

Strategic,  Operational 

Table  8-1.  Reviewed  Systems 

♦TENCAP  -  Tactical  Exploitation  of  National  Capabilities 


Users,  developers,  and  trainers  were  interviewed  with  respect  to  now  anaiysis  was 
affected  by  the  existence  of  these  systems.  The  intent  was  not  to  study  the  per¬ 
formance  of  these  systems  but  to  understand  how  automated  supports  relate  to 
Intelligence  analysis. 

The  observations  from  these  site  visits  and  interviews  were  integrated  with  our  own 
experience  with  automated  systems  and  compared  with  the  implications  of  the  cogni¬ 
tive  model.  As  a  result,  we  have  compiled  a  set  of  conclusions  about  common  design 
problems  in  existing  systems  and  implications  for  future  automated  systems. 


8.1  Common  Problems 

In  the  past,  automated  systems  have  been  developed  with  the  objective  of  exploit¬ 
ing  some  form  of  data  processing  functionality  (memory,  computation,  communica¬ 
tions)  without  a  clear  understanding  of  how  the  system  will  affect  the  performance 
aspect  of  analysis.  Many  of  the  problems  these  systems  have  experienced  in  the 
area  of  user  acceptability  appear  to  be  traceable  to  the  lack  of  understanding  of  the 
human  performance  aspects  of  Intelligence  analysis  on  the  part  of  the  system 
developers.  In  looking  at  the  Information  architecture  of  these  systems  in  relation  to 
the  cognitive  frame''  rk,  there  are  several  common  Inconsistencies  that  may  be  the 
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source  of  usability  problems.  Four  of  the  most  Important  inconsistencies  are: 

•  The  structural  features  of  the  automated  data  base  do  not  match  the  structure 
of  the  analyst’s  threat  model. 

•  The  user  Interface  dialog  does  not  provide  for  separating  the  contexts  of  dif¬ 
ferent  users  in  different  roles  or  different  objectives  In  multiple  anal,  sis  tasks. 

•  Display  and  control  arrangements  are  not  tailored  to  various  thinking  skills  or  to 
variations  in  skill  within  the  user  population. 

•  Feedback  mechanisms  between  information  sources,  analysts,  and  product  users 
are  not  Integral  to  the  system  architecture. 

•  Communication  mechanisms  in  automated  systems  do  not  fully  exploit  the  value 
of  shared  conceptual  models. 

These  problem  areas  are  further  defined  In  the  following  paragraphs. 


8.1.1  INCONSISTENCIES  WITH  THREAT  MODEL 

The  threat  model  is  the  analyst’s  means  of  seeing  the  battlefield  through  an  abstrac¬ 
tion  of  perceived  reality.  The  threat  model  is  an  integiating  framework  that  facili¬ 
tates  multi-source  exploitation  and  multi-disciplinary  analytical  team  effort.  Most 
database  aids  in  automated  intelligence  support  systems  address  some  aspect  of 
threat  model  information  (e.g.,  order  of  battle,  operations  plan,  terrain,  weather). 
Current  systems  have  database  structures  that  are  inconsistent  with  the  threat 
model  in  one  or  more  of  the  following  ways: 

•  The  data  cannot  be  tailored  to  the  mission  context. 

Information,  such  as  order  of  battle,  is  sensitive  to  the  mission  context  and 
environmental  factors.  Multiple  representations  or  adaptivity  features  are 
required  to  tailor  the  threat  model  data  to  a  particular  situation. 

a  The  data  cannot  be  easily  related  to  a  specific  time-space  framework. 

All  threat  model  information  must  be  placed  in  the  time-space  framework  relevant 
to  the  mission. 

•  Data  are  not  classified  as  hypothesis  vs.  observation  (facts,  evidence). 

The  entire  threat  model  is  a  hypothe+ical  structure.  The  hypothetical  structure 
provides  the  means  of  interpreting  new  information.  The  hypothetical  structure 
should  be  distinguishable  from  the  information  gathered  to  substantiate  or 
repudiate  the  hypothesis. 

•  Data  cannot  be  selectively  displayed  (using  geographic  boundaries,  time  window, 
entity  class,  or  entity  parameters)  as  selection  criteria. 

The  totality  of  threat  model  data  requires  that  mechanisms  be  used  for 
decluttertng  the  user’s  view  of  the  threat  model.  Selective  retrieval  and  display 
is  a  required  for  effective  use  of  a  threat  model  database. 

**  Data  cannot  be  portrayed  from  different  perspectives. 
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The  threat  model  has  three  major  perspectives  --  WHITE,  RED,  and  BLUE.  In 
addition,  specific  uses  of  threat  model  information  may  require  varying  levels  of 
detail  or  resolution.  The  ability  to  shift  perspectives  enables  the  analyst  to  deal 
with  much  more  Information  than  Is  possible  with  a  single  perspective. 

•  Information  cannot  be  dealt  with  in  sets. 

Within  the  battlefield,  there  are  numerous  objects  that  can  be  treated  as 
members  of  a  class  (e.g.,  type  of  unit  or  equipment)  or  set  (e.g.,  force  composed 
of  individual  units,  communication  net,  command  structure,  events  occurring  in 
line  of  communication).  Databases  must  be  able  to  recognize  the  association  of 
objects  or  information  events  as  part  of  a  set. 


8.1.2  LACK  OF  MULTI-PROCESSING  SUPPORT 

Automated  systems  in  general  are  not  structured  to  support  the  multiple  processes 
of  the  total  analytical  procedure.  On  the  surface,  analytical  procedure  appears  to  be 
data  d-iven.  Most  ADP  systems  have  assumed  a  data-driven  interaction,  either  by 
Incoming  raw  Intelligence  messages  or  by  commands  from  the  user.  The  IMTIA  cogni¬ 
tive  model  has  shown  the  analytical  process  to  be  objective  driven,  but  with  interr¬ 
uptions  driven  by  opportunities  to  exploit  information  sources.  For  an  automated  sys¬ 
tem  that  hopes  to  follow  and  aid  the  analytical  procedure,  there  must  be  provisions 
for  supporting  multiple  on-going  analytical  processes  that  run  concurrently  and  that 
Interrupt  each  other. 

Because  most  system  designers  have  not  been  aware  of  a  common  structure  In  the 
analytical  process,  there  has  been  no  attempt  to  support  that  structure  or  the 
processes  that  occur  within  that  structure. 


8.1.3  COMPENSATION  FOR  USER  SKILL  LEVEL 

Little  attention  has  been  given  in  current  systems  to  varying  skill  levels  in  the  user 
population.  Intelligence  analysts  represent  many  different  skill  levels  because  of 
personnel  rotation,  ranx,  discipline,  education  and  experience.  A  system  designed  for 
the  casual  user  may  be  cumbersome  and  frustrating  for  the  experienced  user.  A 
system  that  can  only  be  operated  by  an  experienced  user  places  inordinate  demands 
on  training  resources  and  risks  failure  under  the  stresses  of  battlefield  or  crisis 
situations. 

Systems  that  deal  with  the  issue  of  varying  user  skill  levels  are  more  difficult  to 
design  and  guidelines  for  dealing  with  this  issue  have  not  been  available. 


8.1.4  LACK  OF  FEEDBACK  MECHANISMS 

The  IMTIA  study  clearly  identified  the  role  of  feedback  in  the  communication 
processes  between  the  analyst  and  the  intelligence  user  and  between  the  analyst 
and  Information  sources.  Because  most  of  the  feedback  channels  ate  Informal  in 
non-automated  intelligence  production  systems,  the  feedback  mechanisms  may  have 
been  overlooked  by  the  developers  of  automated  systems.  Feedback  Is  an  essential 
feature  in  controlling  information  werload  from  collection  system  inputs  and  in  the 
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generation  of  quality  Intelligence  products.  Feedback  Is  the  natural  mechanism  that 
the  analyst  uses  In  tailoring  intelligence  products  to  the  specific  needs  of  user. 
Because  feedback  mechanisms  operate  over  a  long  period  of  time,  its  importance  is 
not  apparent  In  systems  with  stable  configurations  and  operating  procedures.  How¬ 
ever,  In  a  dynamically-changing  battlefield  environment,  crisis  situation,  or  in  newly 
formed  analytical  teams,  the  criticality  of  feedback  Is  more  apparent. 


8.1.5  COMMUNICATIONS  WITHOUT  SHARED  CONCEPTUAL  MODELS 

The  most  critical  problem  in  tactical  intelligence  production  is  communications.  Tacti¬ 
cal  communications  are  vulnerable  to  jamming,  environmental  factors,  fires,  overload¬ 
ing,  and  disruption  during  maneuver.  Because  of  these  factors,  tactical  communica¬ 
tion  channels  do  not  have  predictable  bandwidths,  availability,  reliability,  and 
throughput  times.  Strategic  communications  as  well  may  be  affected  by  delays 
caused  by  peak  period  overloads,  manual  handling,  and  misdissemination  of  mes¬ 
sages.  None  of  the  AOP  systems  reviewed  addressed  the  problem  of  achieving 
effective  communications  under  these  conditions. 

The  observation  of  informal  communications  in  analyst  activities  has  shown  that  there 
are  natural  mechanisms  for  dealing  with  the  problems  of  unreliable  communication 
channels.  These  mechanisms  exploit  shared  conceptual  models  as  discussed  in 
Chapter  3.  Automated  communications  have  not  yet  been  designed  for  exploiting  the 
concept  of  shai_d  conceptual  models.  Common  problems  are  that  more  data  are  sent 
than  needed,  or  critical  data  needed  to  establish  a  context  for  interpretation  are 
missing. 

Communications  designs  do  not  use  a  cognitive  framework  for  deciding  the  relative 
Importance  of  Information;  under  adverse  communication  conditions  the  most  impor¬ 
tant  information  does  not  always  get  sent  first. 


8.2  Reviews  of  Intelligence  ADP  Systems 

The  reviews  of  automated  systems  conducted  under  the  IMTIA  study  weie  not  aimed 
at  evaluating  the  performance  of  the  automated  data  processing  functions.  The 
objective  was  to  look  for  successes  and  failures  that  were  related  to  human  cogni¬ 
tive  processes. 

Current  systems  are  designed  with  the  primary  objective  of  exploiting  sensor,  com¬ 
munication,  or  adp  technology  rather  than  aiding  the  cognitive  processes  of  the  user. 
User  Interface  configurations  are  designed  in  an  ad  hoc  manner  and  do  not  reflect  a 
systematic  application  of  design  guidelines  (primarily  because  adequate  guidelines 
have  not  been  available).  As  such,  instances  where  automated  systems  successfully 
support  the  cognitive  processes  of  the  analyst  are  largely  due  to  the  intuition  of  the 
designer  or  have  evolved  with  operational  experience. 

Fallings  in  many  cases  could  have  been  avoided  with  the  application  of  guidelines 
that  addressed  the  issues  of  cognitive  performance. 

A  synopsis  of  the  system  reviews  Is  provided  below  to  Illustrate  some  of  the  critical 
Issues  In  future  system  developments. 
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8.2.1  BETA 


BETA  was  designed  as  a  testbed  system  to  demonstrate  the  utility  of  multi-source 
Intelligence  exploitation  ir  targeting  and  situation  assessment. 

One  of  tha  most  Important  features  of  BETA  Is  Its  capability  to  allow  users  to  selec¬ 
tively  call  up  and  display  information  from  the  shared  database.  Each  user’s  display 
can  be  scaled  to  the  geographic  area  of  interest  and  can  display  battlefield  entitles 
selected  by  by  class,  by  time  parameters,  and  by  specific  attributes  of  the  entity. 
The  selective  display  features  are  controlled  by  the  user  through  the  use  of  query 
structures  that  act  on  a  shared  database. 

In  essence,  the  BETA  user  display  Is  adaptive  to  the  role  and  Interests  of  the  user. 
The  BETA  user  command  language,  however,  is  not  adaptive  to  the  role  and  skill  level 
of  the  user.  Although  the  use  of  menus  and  forms  makes  It  possible  for  a  relatively 
Inexperienced  user  to  exercise  the  system  functionality,  the  interactive  dialog  can 
bo  very  cumbersome  for  the  experienced  user. 

BETA  lacks  desirable  feedback  mechanisms  on  what  the  automatic  correlation  algo¬ 
rithms  are  doing  and  lacks  easily  modifiable  templates  for  entities  represented  by 
the  system. 

An  important  feature  of  the  BETA  system  is  that  it  provides  graphics  communications 
for  convening  the  results  of  target  analysis  and  situation  assessment. 

Many  of  t.  e  functions  of  BETA  map  into  the  idealized  threat  model  structure 
presented  in  Chapters  8  and  7.  The  most  important  feature  missing  from  BETA’S 
threat  modeling  capability  is  the  lack  of  time  snapshot  partitioning.  The  user  is 
unable  to  "back  up"  the  displayed  time  window  or  "project"  a  future  time  snapshot. 
This  missing  feature  makes  it  impossible  for  the  user  to  go  back  and  reinterpret  sen¬ 
sor  reports  against  a  new  hypothesis  or  to  project  an  outcome  of  a  situation. 


8.2.2  TCAC 

Very  little  information  was  available  on  TCAC  at  the  time  of  the  IMTIA  system 
reviews.  Comments  made  by  analysts  during  interviews  reflected  a  common  concern 
about  TCAC’s  lack  of  a  geographic  framework  as  an  Integral  part  of  its  display  capa¬ 
bility.  The  implications  of  the  threut  modeling  study  are  clear  about  the  need  for  a 
geographic  framework  in  all  forms  of  tactical  and  strategic  intelligence  analysis. 


8.2.3  fTEP 

ITEP  is  a  product  of  the  TENCAP  Program.  TENCAP  (Tactical  Explication  of  National 
Capabilities),  is  a  p.ogram  designed  to  provide  access  to  National  Ti  chnicai  Means 
intelligence  products  at  the  tactical  level.  ITEP  Is  an  interim  system  that  was 
designed  to  exploit  EUNT  Intelligence. 

Clearly,  the  rrust  Important  Issue  demonstrated  by  ITEP  is  the  need  for  the  analyst’s 
Involvement  In  the  system  development  process.  ITEP  is  regarded  by  users  as  a 
very  successful  development  effort.  Much  of  ITEP’s  success  is  attributed  to  the 
heavy  Involvement  of  intelligence  analysts  In  evolving  the  functional  features  and 
interactive  capabilities.  The  user  Interface  Is  highly  interactive  and  the  analyst  is 
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Involved  In  all  analytical  decisions. 

The  user  interface  is  not  designed  for  a  casual  user. 


8.2.4  DITB 

OITB  is  another  product  of  the  TENCAP  program  that  Is  aimed  at  imagery  exploitation. 
This  system  demonstrates  the  importance  of  sharing  information  on  collection  plans 
and  requirements  In  order  to  fully  exploit  intelligence  gathering  resources.  DITB  as 
well  as  ITEP  demonstrate  the  utility  of  reducing  the  time  delays  in  dissemination  of 
collected  intelligence. 


8.2.5  AUTOMATED  MESSAGE  HAN  DUNG 

The  bulk  of  raw  intelligence  data  and  many  Intelligence  products  are  carried  through 
the  media  of  digital  communication  networks  as  electrical  messages.  These  networks 
also  carry  requirements,  queries,  and  responses  as  messages.  Automated  message 
handling  capabilities  were  introduced  into  the  intelligence  environment  during  the 
70’s  to  deal  with  the  problems  of  increasing  message  volumes,  manual  handling 
delays,  crisis  peak  loading  conditions,  new  requirements  from  new  collection  capabili¬ 
ties,  and  need  for  more  rapid  and  accurate  dissemination. 

Automated  message  handling  systems  also  make  it  possible  to  provide  direct  updates 
to  Intelligence  community  databases  such  as  DIAOLS/COINS. 

Automated  message  handling  capabilities  were  first  introduced  at  the  CIA  and  DIA 
and  subsequently  to  military  commands.  AMH  is  gradually  being  introduced  into  the 
tactical  environment  starting  with  Echelons  above  Corps. 

The  most  Important  cognitive  aspect  of  these  automated  message  handling  systems 
In  intelligence  is  that  dissemination  is  controlled  by  user  Interest  rather  than  by  dis¬ 
tribution  list  assigned  by  the  sender.  Dissemination  control  Is  exercised  at  tne 
receiving  end  rather  than  the  transmitting  end.  This  is  an  extremely  important 
characteristic  of  Intelligence  distribution  that  facilitates  the  exploitation  of  all 
sources  of  Information. 

Intelligence  analysts  need  to  be  in  control  of  the  information-gathering  process.  The 
ability  to  select  information  from  the  electrical  message  carrying  networks  by 
automatic  filtering  is  a  critical  capability  required  to  cope  with  information  overload  in 
the  modern  intelligence  environment. 


8,3  Future  Implications 

The  results  of  the  IMTIA  study  have  direct  implications  for  the  design  of  future 
automated  intelligence  support  systems.  The  general  implications  are: 

•  The  threat  model  can  be  used  as  a  guideline  for  the  organization  of  database 
structures  to  support  the  intelligence  analyst. 

•  The  steps  required  to  build  the  threat  model  can  be  used  as  a  checklist  for  func¬ 
tions  required  to  support  the  storage  and  retrieval  of  Intelligence  data. 
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•  The  design  of  Interactive  dialogs  must  take  into  account  the  multiplicity  of 
analysis  tasks.  Each  analysis  task  carries  with  it  a  different  objective  differing 
Information  needs,  and  differing  procedure  requirements. 

•  Analysis  tasks  adapt  with  the  changing  mission  needs.  Automated  aids  that  sup¬ 
port  the  analyst  in  meeting  mission  information  requirements  must  be  adaptable 
to  the  mission  parameters. 

•  The  nine  thinking  skills  in  the  cognitive  procedure  provide  a  framework  for  the 
design  of  an  interactive  dialog  between  the  analyst  and  the  automated  support 
system.  Each  skill  provides  a  focal  point  for  the  design  of  display  and  control 
features  of  the  user  interface.  The  general  sequence  in  which  these  skills  are 
performed  provider  a  prototypical  order  for  automatic  sequencing  of  machine- 
initiated  help  or  cognitive  aids.  Sequences  can  be  named  and  IdenfTied  with  the 
context  of  a  particular  task/mission  so  that  sequences  can  bo  interrupted, 
saved,  resumed,  or  repeated  automatically. 

•  The  IMTIA  model  of  communications  is  a  mode1  of  normal  inforuial  communications. 
The  analyst  in  day-to-day  activities  uses  shared  conceptual  models  as  a  foun¬ 
dation  for  communications.  Informal  communications  are  extremely  flexible  in 
adapting  to  media  and  time  constraints. 

Day-to-day  interaction  between  parties  establishes  a  broad  base  of  shared  con¬ 
ceptual  models.  In  informal  communications,  once  a  context  has  been  esta¬ 
blished,  the  actual  Information  exchange  can  be  very  brief  but  achieve  a  high 
level  of  understanding.  This  Is  especially  Important  when  the  time  available  Is 
extremely  limited. 

Automated  systems  can  be  similarly  designed  by  organising  Information  into 
context-specific  networks  (e.g.,  artillery  targeting,  EW  targeting,  weather, 
OPSEC,  etc.)  Many  context-specific  networks  can  be  mapped  onto  a  single  digi¬ 
tal  message  network.  Modern  communications  protocols  can  be  utilized  for  error 
control  and  allocating  available  bandwidth  between  the  multiple  networks. 

Automated  communications  should  be  based  on  exploiting  the  nature  of  shared 
conceptual  models  In  order  to  achieve  understandability  and  effectiveness. 
Designers  must  recognize  the  multiplicity  of  the  Information  networks  that  per¬ 
meate  Intelligence  operations,  each  using  different  conceptual  models  of  the  real 
world.  Because  of  the  multiple  contexts  in  which  the  same  information  may  be 
used,  communications  designers  must  recognize  the  need  to  tailor  information  to 
a  particular  user  or  usage.  Although  the  number  of  logical  networks  may  be  more 
numerous  under  this  approach,  the  actual  data  transfer  rates  can  be  minimized 
by  exploiting  existing  shared  knowledge  between  sending  and  receiving  parties 
that  is  context  dependent.  Communications  systems  that  do  not  exploit  context 
must  transmit  substantially  more  data  in  each  message  to  convey  the  same 
amount  of  information. 

The  use  of  a  cognitive  framework  for  the  design  of  the  user  interface  is  being  pur¬ 
sued  under  an  on-goinjj  research  project  sponsored  by  ARI.  (Research  on  Human 
Factors  in  Design  for  C  I,  Contract  No.  MDA903-81-C-0579.)  This  research  effort  is 
aimed  at  developing  guidelines  for  system  developers  who  wish  to  Incorporate  adap¬ 
tive  user  Interface  features. 
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0.  RESEARCH,  DEVELOPMENT,  AND  TRAINING  ISSUES 


Tha  eight  previous  chapters  have  summarized  and  discussed  current  cognitive  Issues 
In  performing  Intelligence  analysis  Some  of  those  issues  have  direct  implications  for 
training;  others  raise  questions  that  require  further  research  and  development. 
Further  research  would  in  turn  provide  clarification  and  understanding  that  would  be 
applicable  to  improved  training  methods. 


0.1  Cognitive  Processus 

Some  Impoitant  problems  raised  in  analyzing  and  evaluating  cognitive  processes  and 
performance  in  intelligence  analysis  are  described  below. 


9.1.1  TASK  SEGMENTS  AND  SKILLS 

While  the  task  segments  underlying  analytic  performance  have  been  identified,  the 
actual  cogn'tive  skills  required  to  execute  the  task  segments  have  not  been  itemized 
Individually.  The  general  skills  described  In  Chapter  4  apply  to  all  task  segments.  As 
described  in  Section  7.1,  it  is  hypothesized  that  there  exist  very  specific  cognitive 
skills  pertinent  to  the  individual  task  segments.  These  should  be  idenlified  individu¬ 
ally  and  training  materials  developed,  tailored  to  the  individual  task  segments. 

The  different  task  segments  are  associated  with  different  biases  and  are  differen¬ 
tially  affected  by  the  cognitive  biases  discussed  in  Section  2.4.  The  relationship 
between  task  segments  and  biases  should  be  investigated.  Such  an  Investigation 
could  begin  with  a  case  study  review  of  intelligence  products  to  identify  where  and 
how  cognitive  biases  might  have  led  to  misleading  or  erroneous  predictions  or  situa¬ 
tion  assessments. 


9. 1 .2  TRAINING  IMPLICATIONS  FROM  THE  COGNITIVE  MODEL 

The  various  aspects  of  the  cognitive  model,  as  described  In  Chapters  2  through  5 
have  numerous  implications  for  training. 

Bee  use  of  the  importance  of  decision  making  in  intelligence  analysis,  it  is  imperative 
tha  nore  research  be  devoted  to  the  types  of  decisions  that  analysts  have  to  make, 
how  they  make  them,  and  how  they  affect  the  intelligence  product.  This  would  be  a 
high  pay-off  area  for  research,  sinct  analytical  thinking  covers  an  extremely  broad 
and  complex  domain.  Though  much  .  j  known  about  decision  theory  and  rules  for 
application,  it  would  be  detrimental  to  train  analysts  using  extant  knowledge  in  deci¬ 
sion  theory  and  by  providing  them  with  a  few  formal  rules  without  first  investigating 
the  context  and  contents  of  analytic  decisions. 

While  there  exist  numerous  automated  and  non-automated  decision  aids,  many  of 
these  aids  are  not  appropriate  for  intelligence  analysis  because  the  quantity  and 
quality  of  Information  necessary  to  utilize  the  aids  in  the  intelligence  arena  is  not 
available.  However,  there  exist  certain  recurrent  problems  encountered  when  mak¬ 
ing  decisions  in  the  intelligence  context  that  are  tied  to  llmltatlbns  of  the  human 
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Information-processing  system,  and  there  are  fundamental  procedures  that  would 
prove  useful  In  dealing  witli  those  problems. 

The  development  of  effective  analytical  thinking  Is  likely  to  proceed  through  experi¬ 
ence  with  relevant  classes  of  examples.  From  such  experience  will  emerge  an 
awareness  of  common  pitfalls  Inherent  In  analysis  as  well  as  procedural  guides  and 
decision  paths  to  maximize  the  quality  of  performance.  Towaid  this  goal,  a  selective 
list  of  fundamental  concepts  and  problem  areas  in  the  context  of  operational  intelli¬ 
gence  could  be  prepared  for  inclusion  in  a  training  program.  Also,  a  limited  set  of 
examples  could  be  developed  such  that  useful  guides  (procedures)  tor  dealing  with 
the  problem  areas  can  be  illustrated  and  imparted  effectively  to  the  analyst 
trainees.  The  suggested  procedures  would  serve  to  develop  a  general  attitude 
about  problem  solving  and  decision  making  that  is  conducive  to  optimizing  analytical 
thinking.  Although  the  trainee  may  never  encounter  the  precise  events  described  in 
the  examples,  experience  with  the  important  problem  areas  and  useful  modes  of 
solution  (the  analytical  processes)  should  generalize  to  a  broad  class  of  similar  situa¬ 
tions. 

Among  the  concepts  and  problem  areas  In  analytical  tnlnklng  that  should  be  con¬ 
sidered  for  Inclusion  In  a  training  program  are: 

•  Inflexibility  of  thought  (cognitive  entrenchment,  e.g.,  confirmation  bias). 

•  Separation  of  relevant  from  irrelevant  information  (e.g.,  unwarranted  hypothesis 
switching). 

•  Mitering  biases  (selectivity,  polarization). 

•  Interpretation  of  sparse  or  uncertain  data  (caricature  effect). 

•  Memory  access  shortfalls  (similarity  effects). 

•  Information  management  (summarizing,  sorting,  assessing  trends,  checklists, 
memory  aids). 

«  Fallacies  of  logic  (e.g.,  the  "gambler's  faiiacy”  in  prediction). 

•  Asking  the  right  questions  (recognizing  goals). 

Among  the  general  decision  guides  to  analytical  thinking  that  might  be  addressed  are: 

•  Seeing  the  total  picture  (avoiding  over-focusing  on  details). 

•  Withholding  judgment  (hypothesis  testing  as  an  iterative  process). 

•  Using  models  (doctrine,  templates,  prototypes). 

•  Generating  hypotheses  based  on  partial  information. 

•  Changing  perspective  (restructuring  problems). 

•  Understanding  uncertainty  and  reliability  (will  to  doubt). 

•  Using  stable  substructures. 

•  0l  tcu3Slng  problems  and  decision  alternatives  with  others. 
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9.2  The  Analy.it/Usar  Dialog 

The  Importance  of  communication  in  general,  and  botwoon  analyst  and  user  specifi¬ 
cally,  should  be  explored  more  thoroughly.  Ways  to  optimize  the  development  and 
use  of  shared  conceptual  models  to  enhance  communication  and  reduce  errors  and 
misunderstandings  should  be  Investigated.  There  are  two  parallel  areas  of  inquiry 
relevant  to  communication,  namely  by: 

•  Typos  and  araa3  of  communications  (e.g.,  which  ones  are  most  Important,  which 
ones  might  Increase  danger  If  misunderstandings  occur). 

•  Types  and  areas  of  misunderstandings  that  are  known  to  occur. 

Inquiries  should  begin  wkh  interviews  and  result  in  lists,  hierarchically  organized  by 
importance,  of  these  two  areas.  Among  the  specific  issues  to  be  investigated  are 

the  following: 

•  The  analyst  must  have  an  adequate  understanding  of  how  the  Intelligence  pro¬ 
duct  will  affect  the  user’s  perception  of  threat.  We  know  that  the  desired  reac¬ 
tion  Is  for  the  user  to  perceive  an  Increased  level  of  control  and  a  reduction  in 
danger.  The  first  research  question,  therefore,  concerns  the  measurement 
requirements  for  determining  any  changes  in  the  analyst’s  and  the  user’s  per¬ 
ception  of  threat. 

•  Current  feedback  to  analysts  is  generally  informal  or  non-existent  unless  the 
analyst  is  In  direct  contact  with  the  user.  Feedback  mechanisms  are  required 
for  the  analyst  to  know  if  these  effects  are  being  achieved.  The  following  ques¬ 
tions  sir  Id  be  Investigated: 

—  Whai  are  optimum  feedback  mechanisms? 

—  How  can  they  be  exploited? 

—  How  can  their  effectiveness  be  measured? 

•  Shared  conceptual  models  have  the  potential  tor  being  exploited  to  reduce 
errors  and  costs  of  battlefield  communications,  as  well  as  for  improving  the 
commander’s  timeline  for  control.  Ways  of  measuring  the  efficiency  of  SCMs  are 
needed  to  Justify  revamping  current  communications  concepts. 

There  are  several  research  questions  that  deal  directly  with  the  way  SCMs,  as 
well  as  CMs,  are  generated  and  used  in  information  processing,  analysis,  and 
communications: 

—  What  are  the  characteristics  of  the  cues  that  allow  for  the  "best"  (most 
complete,  most  appropriate)  retrieval  from  external  memory? 

—  What  knowledge  items  do  j  need  within  our  own  CMs  in  order  to  make  use 
of  external  memory? 

—  What  are  the  best  retrieval  cues  to  access  other  CMs  within  one’s  own 
memory  or  to  access  external  memory?  Are  they  the  same? 

—  What  types  of  Informational  Items  have  to  be  shared  for  SCMs  to  be  optimally 
offective?  Intuitively,  one  might  suggest  the  following  as  Important  common 
factors  for  effective  SCMs: 
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—  Context,  framework. 

—  Goals. 

—  Language. 

—  Affective  value  of  CM. 

—  How  can  affective  SCMs  be  generated? 

—  How  can  the  effectiveness  of  SCMs  be  measured"7 

If  some  of  these  questions  could  be  determined,  then  training  materials  could  !>e 
structured  so  as  to  Include  appropriate  Information  to  generate  SCMs  and 
appropriate  retrieval  cues  for  correct  access  to  internal  and  external  memory. 

Summary  Questions  on  research  Involving  SCMs  Include: 

•  How  are  SCMs  established  and  can  the  process  be  speeded  up? 

•  How  can  the  shared  aspects  of  conceptual  models  be  Identified? 

•  How  can  areas  of  misunderstanding  be  identified? 


9.2.  T  IMPLICATIONS  FOR  TRAINING 


The  views  presented  here  concerning  CMs,  SCMs,  and  goals,  have  several  important 
Implications  for  understanding  learning,  retention,  and  recall,  and  hence,  for  the 
development  of  training  materials,  for  education,  for  training  new  skills,  for  the 
maintenance  of  skills,  and  for  improving  communications  within  the  Intelligence  com¬ 
munity. 

Some  of  these  implications  are  as  follows: 


•  It  is  important  to  develop  a  common  framework  among  analysts  concerning  the 
guais  of  analysis,  Its  organizational  basis,  and  its  role  within  the  military  commun¬ 
ity  and  for  the  overall  goal  of  national  defense.  This  framework  should  be  shared 
at  all  levels  of  analysis  (horizontally  and  vertically)  and  it  should  be  shared  with 
the  users. 


Developing  such  a  framework  has  two  consequences- 

—  It  provides  the  new  analyst  with  an  organized  structure  (a  new  CM  in 
memory)  within  which  to  str  re  new  learning  materials.  The  alternative  is  that 
new  materials  must  be  stored  in  existing  CMs,  carried  over  from  earlier 
training.  These  existing  CMs  may  be  quite  inappropriate  for  organizing  new 
materials,  and  the  result  is  confusion  and/or  slower  learning. 

—  It  sets  the  context  for  establishing  SCMs  between  analysts,  and  it  provides 
the  basis  for  the  analysts  themselves  to  establish  SCMs  with  their  clients. 

•  The  goals  and  subgoals  of  analysis  should  be  clearly  spelled  out  and  Invested 
with  affective  values  so  as  to  Increase  the  importance  (and  hence,  the  speed  of 
learning  and  ultimate  performance)  of  the  CM  that  is  being  established. 

•  Once  the  framework  has  been  established,  the  training  materials  to  be  presented 
should  always  be  related  to  that  framework.  Analysts  should  understand  how 
hypothesis  generation,  for  example,  is  related  to  the  production  task,  the  mission 
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requirement,  and  the  goals  of  analysis. 

•  Access  to  existing  CMs:  It  is  important  to  relate  new  materials  to  items  that  are 
already  In  memory.  For  example,  If  an  analyst  has  a  good  background  in 
mathematics  and  statistics,  It  is  Important  to  Insure  that  the  connection  is  made 
between  the  new  material  to  be  taught  and  the  relevant  background  knowledge. 
In  other  words,  training  materials  must  be  developed  based  on  an  understanding 
of  the  trainee’s  available  CMs  and  a  clear  Identification  of  the  objectives  and 
the  goals  of  training.  The  taxonomy  of  knowledge  described  in  Section  3.3 
should  be  expanded  to  Include  individual  analysts’  existing  knowledge  bases  and 
the  required  knowledge  categories  for  optimum  performance. 

•  Training  and  knowledge  maintenance  must  address  the  ptoblem  that  analysts 
have  when  they  change  jobs  or  when  they  are  transferred  from  one  theater  of 
operation  to  another.  The  descriptions  and  attributes  of  CMs  (Section  3.2)  sug¬ 
gest  ways  to  make  such  changes  easier  for  the  analyst  and  more  effective  for 
meeting  production  requirements. 

•  The  views  concerning  the  nature  and  characteristics  of  CMs  should  be  con¬ 
sidered  when  developing  automated  databases  as  aids  to  analysis. 


8.3  Goal  Orientation 

The  1MTIA  cognitive  model  emphasizes  the  Importance  of  a  context-specific  goa! 
orientation  in  guiding  analytic  performance.  Analysts  must  know  their  goals  and  share 
goals  with  other  members  of  the  Intelligence  community  to  fulfill  mission  requirements 
affectively  At  the  same  time,  these  goals  must  be  explicit  for  the  ideal  product  to 
be  effective  as  an  evaluation  tool.  Some  research  Issues  are  discussed  below. 


9.3. 7  LEARNING  GOALS 

Analysts  should  be  encouraged  to  learn  how  to  identify  their  goals  arid  how  to  use 
goals  in  structuring  their  tasks  and  future  training  requirements.  Without  goal  direc¬ 
tion,  some  analysts  may  have  a  great  deal  of  difficulty  in  determining  what  is  impor¬ 
tant  for  them  to  know  at  any  given  time  or  how  to  process  what  they  know.  One  way 
to  increase  the  likelihood  that  analysts  will  learn  and  aggregate  the  appropriate  data 
elements  in  an  efficient  manner  is  for  them  to  adopt  or  be  provided  with  explicit 
learning  goals. 

Learning  goals  might  take  the  form  of  questions,  or  they  might  simply  be  statements 
to  "learn  about  X".  in  addition,  the  goals  could  be  stated  generally  (e.g.,  "learn 
about  the  overall  threat  of  enemy  forces  in  Sector  X"),  or  they  could  refer  to 
specific  bits  of  information  (e.g.,  "learn  about  the  movement  of  maneuver  units  in 
Section  X").  The  more  specific  the  learning  goal,  the  greater  the  chance  that  the 
analyst  will  be  successful  In  mastering  the  goal. 

The  use  of  goals  in  learning  complex  materials  demonstrates  that  learning  goals 
induce  the  learner  to  process  the  material  In  such  a  way  that  performance  on  test 
questions  (usually  sentence  completion  Items)  referring  to  the  goal-relevant  material 
Is  Improved.  This  Improvement  cannot  be  explained  solely  as  a  redistribution  of  pro¬ 
cessing  time.  The  extent  of  the  improvement  Is  somewhat  dependent  upon  the 
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number  of  goals  to  be  mastered  and  the  ease  with  which  the  learner  can  locate  the 
appropriate  material  In  a  text.  With  a  greater  number  of  goals,  most  subjects  take 
longer  to  study  the  material  and  they  are  less  likely  to  learn  the  information  relevant 
to  each  goal.  If  all  of  the  data  that  are  relevant  to  a  particular  goal  are  not  located 
together  In  the  Information  flow,  then  It  will  sometimes  be  the  case  that  only  the 
Information  contained  in  the  first  reference  to  the  goal-relevant  data  will  be 
thoroughly  studied  (Gagne  &  Rothkopf,  1975).  Therefore,  there  are  some  limiting 
factors  In  adopting  learning  goals  as  learning  guides,  and  the  limits  are  dependent 
upon  both  the  learner  and  the  materials.  The  available  basic  research  suggests  that 
each  analyst  should  (a)  adopt  only  a  limited  number  of  goals  to  guide  performance 
and  (b)  acknowledge  potential  Interpretive  biases  caused  by  concentrating  too 
heavily  on  the  initial  information  pertinent  to  the  goals.  However,  no  research  exists 
on  iearning  improvements  with  multiple  goals  when  those  goals  are  hierarchically 
organized,  as  proposed  In  Section  5.1.  It  is  likely  that  multiple  goals,  when  hierarchi¬ 
cally  organized,  will  enhance  learning  rather  than  Impede  it.  This  might  be  a  fruitful 
and  interesting  area  for  investigation. 

9.3.2  SHARED  GOALS 

In  developing  shared  conceptual  models  and  in  Identifying  relevant  goals,  a  connec¬ 
tion  must  be  made  between  the  goals  that  are  to  be  shared  and  goals  that  are 
already  important  to  the  individual.  Individuals  have  different  goals,  but  to  optimize 
teamwork,  there  should  be  some  shared  goals  at  some  level  of  the  hierarchy.  Both 
during  training  and  in  the  work  setting,  it  Is  suggested  that  the  common  goals  of 
analysis  be  made  personally  important  for  each  analyst. 

Shared  goals  promote  the  development  of  SCMs.  Though  analysts  need  not  accept 
the  users*  goals  as  their  own,  they  do  need  to  know  and  understand  them. 

For  purposes  of  training  and  improvement  of  analytic  performance,  It  is  necessary  to 
Identify  a  hierarchy  of  goals,  subgoals,  and  tasks  and  to  relate  this  hierarchy  to  the 
cognitive  skills  required  to  perform  the  task. 

9.4  Issues  Related  to  Threat 

Threat  is  ono  of  the  primary  conceptual  issues  that  intelligence  analysts  deal  with.  A 
threat  model  has  been  developed  by  Logicon  to  serve  as  a  shared  conceptual  model 
between  analysts  and  users,  and  to  provide  a  basis  for  making  more  accurate  intelli¬ 
gence  evaluations  and  predictions.  Much  research  remains  to  be  done,  however,  for 
the  purposes  of  the  threat  model  to  become  fully  realized. 

Research  related  to  threat  can  be  divided  into  two  categories: 

1.  Research  related  to  the  perception  of  threat. 

2.  Research  related  to  the  parameters  of  the  threat  model. 

9.4.1  THREAT  PERCEPTION 

There  are  several  areas  with  potential  payoffs  for  further  research  in  evaluating  and 
measuring  threat  perception. 


1.  The  payoffs  are  in  selecting  optimum  reporting  rates  for  threat  information  to 
ensure  that  the  user  can  react  with  control.  Too  frequent  reporting  may  reduce 
the  significance  of  changes  or  decrease  the  user’s  ability  to  detect  trends.  Too 
infrequent  reporting  may  decrease  the  user’s  ability  to  respond  without  panic. 

Research  in  this  area  would  be  concerned  with  ways  to  measure  the  user’s 
reaction  to  threat  information  as  a  function  of  reporting  rate. 

2.  The  user's  reaction  to  threat  in  general  involves  a  decision  regarding  allocation 
of  resources  for  control.  The  impact  of  uncertainty  is  to  reduce  the  user’s  per¬ 
ception  of  control  and  increase  the  probability  of  errors  in  battlefield  resource 
allocation. 

Research  in  this  area  would  be  directed  at  mechanisms  to  measure  the  user’s 
level  of  uncertainty  as  a  means  of  feedback  to  the  intelligence  production 
operation. 

3.  Assuming  that  the  Milburn  and  Watman  (1981)  model  Is  valid,  it  would  be  useful 
to  devise  means  for  an  objective  evaluation  of  observable  behavioral  responses 
(l.e.,  sense  of  comfort,  challenge,  alienation,  panic)  associated  with  perception 
of  threat  and  control.  Research  questions  would  deal  with  the  differences 
between  such  observable  responses  in  the  strategic  and  the  tactical  battle¬ 
fields.  A  possible  approach  would  be  to  review  intelligence  cases  with  these 
factors  in  mind,  namely  perception  of  threat,  available  physical  control,  and  phy¬ 
sical  responses  associated  with  different  degrees  of  each.  Such  a  review 
should  attempt  to  determine  if  a  correlation  exists  that  would  validate  the  Mil- 
burn  and  Watman  model. 

4.  An  additional  research  area  concerns  the  problems  of  the  extraneous  factors 
that  affect  intelligence  analysis  and  reporting,  as  discussed  abc'.'e  (i.e.,  national 
policy,  user  idiosyncrasies,  etc.).  While  not  strictly  a  problem  of  intelligence 
analysis  per  se,  it  is  obviously  a  source  of  many  poor  anaiyticai  producis. 

The  fact  that  so  many  extraneous  factors  impact  adversely  on  the  intelligence 
product  is  a  matter  of  great  concern  to  observers  of  and  participants  in  the  U.  S. 
Intelligence  community.  The  IMTIA  studies  suggest  that  these  problems  are  pri¬ 
marily  due  to  a  lack  of  shared  conceptual  models,  shared  goals,  and  to  poor  com¬ 
munication.  The  IMTIA  cognitive  model  contains  several  useful  concepts  that,  if 
applied,  can  help  alleviate  these  shortcomings  of  the  intelligence  production 
cycle. 


9.4.2  PARAMETERS  OF  THE  THREAT  MODEL 

As  discussed  in  Chapter  6,  the  threat  model  has  three  major  aspects:  white,  red,  and 
blue.  Each  of  these  is  made  up  of  numerous  elements  that  have  variable  impacts  on 
the  implications  derived  from  the  modal.  The  implications  of  concern  to  intelligence 
analysts  are: 

•  How  to  ass  ass  threat. 

•  How  to  assess  threat  credibility. 

•  How  to  communicate  threat. 


•  Analysts'  and  users’  reactions  to  threat. 

•  How  to  evaluate  threat. 

It  la  possible  that  these  Issues  are  treated  differently  by  analysts  as  they  view  the 
battlefield  from  the  white,  red,  or  blue  perspective,  respectively.  It  might  be 
Intere  sting  to  investigate  this  idea. 

There  would  also  be  a  high  pay-off  value  in  developing  other  notions  underlying  the 
threat  model. 

For  example,  not  all  elements  of  the  threat  model  have  the  same  relative  importance 
for  the  various  analytic  tasks.  Research  in  this  area  would  consist  in  identifying  the 
relative  Importance  of  the  threat  model  elements  for  assessing  situations,  making 
predictions,  or  evaluating  and  dealing  with  uncertainty. 

A  better  understanding  of  how  uncertainty  and  risk  affect  analytic  performance  and 
products  could  make  a  significant  impact  on  training.  For  example,  some  research 
should  be  devoted  to  identifying  different  types  of  uncertainties,  such  as  uncertain¬ 
ties  concerning: 

1.  Currently  existing  physical  structures  (e.g.,  tanks,  enemy  installations). 

2.  Future  physical  structures  (e.g.,  new  weapons). 

3.  Current  non-physical  red  elements  (enemy  doctrine). 

4.  Future  non-physical  red  elements  (enemy  intentions). 

6.  Current  white  elements  (given  inadequate  maps,  for  example). 

6.  Future  white  elements  (e.g.,  weather). 

7.  Current  and  future  blue  elements  (e.g.,  availability  of  resources). 

These  uncertainties  are  categorized  by  "types".  The  question  is,  do  the  types  of 
uncertainties  have  differential  effects  on  the  tasks.  Also,  are  the  types  of  uncer¬ 
tainties  correlated  in  some  way  with  the  judged  degrees  of  uncertainty  that  a  com¬ 
mander  might  have?  That  is,  are  certainties  treated  differentially  depending  on 
type?  Given  certain  degrees  of  un<  :rtainty,  how  are  predictions  affected?  Specifi¬ 
cally,  are  probabilities  assigned  differentially? 

Other  questions  related  to  uncertainty  might  be  asked,  such  as: 

•  What  Is  the  judged  -isk,  given  different  types  of  uncertainties? 

•  How  are  the  probabilities  of  events  treated,  given  differential  judged  risk? 

•  How  do  analysts/commanders  estimate  reliability,  validity,  or  countering  capabili¬ 
ties  of  various  types  of  information? 

All  these  factors  should  be  more  carefully  evaluated,  the  literature  searched  for 
Information  on  these  factors,  and  a  research  program  designed  to  answer  some  of 
the  more  important  questions. 

In  summary,  the  following  issues  related  to  the  threat  model  should  be  investigated: 

•  The  relative  Importance  of  the  parameters  of  the  threat  model  as  they  impact  on 
the  prediction  of  threat. 
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•  The  dynamics  of  the  parameters;  i.e.,  which  factors  change  faster  than  others 
end  how  these  changes  Impact  on  each  other. 

•  The  effects  of  different  degrees  of  uncertainty  on  decision  making  (i.e.,  humans 
tend  to  deviate  from  normative  models  of  decision  making:  are  these  deviations 
a  function  of  the  uncertainties  associated  with  the  different  factors  that  need 
to  be  considered  when  making  decisions?). 

•  Whether  there  are  different  types  of  uncertainty  and  whether  these  types  have 
a  differential  Impact  on  decision  making  and  strategic  predictions.  For  example, 
are  uncertainties  related  to  physical  Items  (e.g.,  existing  enemy  installations) 
treated  differently  than  uncertainties  related  to  hypothesized  behavioral  items 
(e.g.,  future  troop  movements  or  enemy  Intentions)? 

•  Whether  differences  in  uncertainty  types  affect  how  probabilities  are  assigned, 
how  risk  is  perceived,  and  how  validities,  reliabilities,  and  countering  capabilities 
are  estimated. 

•  How  different  degrees  of  uncertainty  affect  the  product  outcome  or  the  report¬ 
ing  of  the  product. 

9.6  The  Ideal  Product 

The  concept  of  the  ideal  product  arose  out  of  the  need  to  define  a  baseline  state  for 
the  cognitive  model.  That  is,  an  assumption  was  made  that  one  can  define  an  ideal 
analyst  performing  ideally  and  producing  an  ideal  product.  This  baseline  state  would 
serve  as  the  evaluation  criterion  for  actual  performances  and  products.  The  differ¬ 
ences  between  the  idea!  and  the  actual  would  be  used  to  identify  training  needs,  as 
well  as  areas  where  maintenance  of  knowledge  or  skills  should  be  focused.  These 
assumptions  should  be  investigated  for  their  validity  and  usefulness. 


9.6  Conclusion 

Several  research  topics  and  training  issues  have  been  discussed.  Uo  specific 
methods  have  been  proposed  for  actually  performing  experiments.  In  general,  how¬ 
ever,  most  Issues  discussed  could  be  subjected  to  controlled  experiments  on  the 
one  hand,  or  could  be  usefully  investigated  by  combining  in-depth  interviews  of  on- 
the-job  Intelligence  analysts  with  results  from  the  existing  cognitive  and  analytic 
literature. 
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